Learning assessment at shoulder surgical skills course

INTRODUCTION

Surgical skills courses are an important part of learning during surgical training. The assessments at these courses tend to be subjective and anecdotal. Objective assessment using multiple choice questions (MCQs) quantifies the learning experience for both the organisers and the participants.

MATERIALS AND METHODS

Participants of the open shoulder surgical skills course conducted at The Royal College of Surgeons of England in 2005 and 2006 underwent assessment using MCQs prior to and after the course.

RESULTS

The participants were grouped as non-consultants (14) and consultant orthopaedic surgeons (8). All participants improved after attending the course. The average improvement was 17% (range, 4–43%). We compared the two groups while adjusting for the association between pre-course score and score gain. We found a strong correlation between pre-course score and score gain (r = 0.734; P = 0.001). Adjusted for pre-course score, we found that the score gain (learning) for the non-consultants was slightly larger than for the consultants, but this did not reach statistical significance (P = 0.247).

CONCLUSIONS

All participants had a positive learning experience which did not have a significant correlation to the grade of surgeon.     

Training and skills for breast surgeons in the new millennium

Breast surgeons are learning to adapt to an evidence-based, guideline-directed and outcome-orientated culture as key members of the multidisciplinary team. Recent data has confirmed the central role of surgery in preventing recurrence, improving survival and reducing risk, and rising scrutiny is raising the standards of breast cancer treatment. Specialization is increasing in breast surgery as a result of foreshortened training, greater patient demand and increasing trainee expectations and breast surgeons are learning new skills. The balance of the breast surgeon's skill-base reflects personal preferences and professional networks, local needs and geographical -constraints, and current developments in advanced surgical training curricula. Modern training programmes need to recognize these needs, supporting interprofessional cross-specialty training initiatives and encouraging professional development. Trainees and -trainers will need to acquire new skills in diagnostic, targeted, oncoplastic and prophylactic procedures through a variety of new training initiatives. Breast surgery is standing on the threshold of change, and breast surgeons must develop new strategies, new skills and new alliances to strengthen their role in this expanding specialty.     

Building a laparoscopic surgical skills training laboratory: resources and support.

BACKGROUND: Technical skills have historically been developed and assessed in the operating room. Multiple pressures including resident work hour limitations, increasing costs of operating room time, and patient safety concerns have led to an increased interest in conducting these activities in a safe, reproducible environment. To address some of these issues, many residency programs have developed laparoscopic surgical skills training laboratories. We sought to determine the current status of laparoscopic skills laboratories across residency programs. METHODS: In December 2004, surveys were mailed to all 251 United States general surgery residency program directors. This brief 2-page survey consists of 9 questions regarding laparoscopic skills training laboratories. RESULTS: Of the 251 mailed surveys, 111 completed surveys were returned (44%). Of the respondents, 81 have laparoscopic skills training laboratories in place (80%). Skills laboratories that used a defined curriculum, and general surgery programs that shared their laboratories with other training programs were determined to have significantly more resources. A wide variety of funding sources have been used to develop and support these skills laboratories. CONCLUSIONS: Significant variability in training practices and equipment currently used exists between laboratories. A more efficient, standardized approach to skills training across residency programs is a desirable goal for the immediate future.     

The emotional intelligence of surgical residents: a descriptive study

BACKGROUND: We assessed educational needs with regard to leadership, communication, and emotional intelligence (EI) among surgical residents. METHODS: General surgery residents (n = 74) were examined using the BarOn Emotional Quotient Inventory (EQ-i) and a 20-item survey. RESULTS: Residents believed that leadership skills were important (mean 4.7, SD .5) and that they had skills in each the five EI areas (overall mean 4.1, SD .8). Both the overall group's EQ-i scores (mean 106.6, SD 11.6), as well as scores on the 20 components of the EQ-i (range of means 102-110), were higher than national norms. Individuals varied substantially on EQ-i subscale scores. CONCLUSIONS: Surgical residents believed that leadership skills are important and scored strongly on both an EI self-assessment and the EQ-i. Specific individual differences in subscale scores can potentially identify areas for direct educational intervention.     

Imagery practice and the development of surgical skills

BACKGROUND: The purpose of this review is to explore the potential role of imagery practice during the acquisition of surgical skills, imagery practice being the mental rehearsal of a skill. METHODS: The core of this review is derived from a literature search of a computer database (Medline). FINDINGS: The cognitive processing that occurs during times of intense learning involves processes such as dream enactment behavior and imagery practice. These processes complement and augment the more usual forms of practice. CONCLUSIONS: Imagery practice provides a mechanism for the explicit learning of surgical skills.     

Assessment of communication, professionalism, and surgical skills in an objective structured performance-related examination (OSPRE): a psychometric study

Background

The purpose of this study was to investigate the reliability and validity of a performance assessment of communication, professionalism, and surgical skills competencies for surgery residents.

Methods

Fourteen residents from the general surgery program of the University of Calgary were assessed in 7 surgical simulation stations that included communication and professionalism skills.

Results

The internal consistency reliability of the checklists and global rating scales combined was adequate for communication (α = .75–.92) and surgical skills (α = .86–.96), but not for professionalism (α = 0). There was evidence of validity as surgical skills performance improved as a function of postgraduate year level but not for the professionalism checklist. Surgical skills and communication correlated in the 2 stations assessed (r = .55 and .57; P < .05).

Conclusions

There is evidence for both reliability and validity for simultaneously assessing surgical skills and communication skills. Further instrument development is required to assess professionalism in a structured examination context.     

Development of a comprehensive vascular skills assessment for surgical trainees

Background

Although vascular skills are important to general surgeons, vascular surgery has become a separate specialty, and therefore, there may be an erosion of vascular skills acquired by general surgical trainees. The purpose of this study was to develop a reliable and valid comprehensive vascular skills assessment (CVSA) of both knowledge and technical skills.

Methods

Twenty-four of 38 general surgical residents at the University of British Columbia completed a two-part CVSA consisting of a written examination and a series of 4 technical stations in a skills laboratory. Technical performance was rated using validated scales.

Results

The mean overall CVSA score was 50%. The CVSA demonstrated construct validity, with improvement in scores with increasing postgraduate year level (P = .01). The overall reliability (Cronbach's α) was .90.

Conclusions

The CVSA developed in this study is a comprehensive assessment of vascular skills that is both valid and reliable. It offers an objective and feasible assessment of general surgical trainees' vascular skills.     

Assessment of technical and nontechnical skills in surgical residents

Background

Surgical competence encompasses both technical and nontechnical skills. This study seeks to evaluate the validity evidence for a comprehensive surgical skills examination and to examine the relationship between technical and nontechnical skills.

Methods

Six examination stations assessing both technical and nontechnical skills, conducted yearly for surgical trainees (n = 120) between 2010 and 2014 are included.

Results

The assessment tools demonstrated acceptable internal consistency. Interstation reliability for technical skills was low (alpha = .39). Interstation reliability for the nontechnical skills was lower (alpha range −.05 to .31). Nontechnical skills domains were strongly correlated, ranging from r = .65, P < .001 to .86, P < .001. The associations between nontechnical and technical skills were inconsistent, ranging from poor (r = −.06; P = .54) to moderate (r = .45; P < .001).

Conclusions

Multiple samplings of integrated technical and nontechnical skills are necessary to assess overall surgical competency.     

Non-technical skills for surgeons (NOTSS)

Modern patient safety is approached in a systematic manner with a view to both minimizing harm and maximizing good practice. Small performance adjustments are used to make incremental gains. Non-technical skills are a key set of skills necessary for safe, effective patient management. The NOTSS (non-technical skills for surgeons) taxonomy defines the skills required of surgeons in the operating room, namely situation awareness, decision making, communication and teamwork, and leadership. Each of these skill categories pertains to a distinct skill set that needs to be practised and refined. Developing understanding of how we perform these skills and identifying tools and techniques to integrate into daily practice allow us to both reduce human error and make our surrounding team performance more effective.     

Non-technical skills for surgeons (NOTSS)

Modern patient safety is approached in a systematic manner with a view to both minimizing harm and maximizing good practice. Small performance adjustments are used to make incremental gains. Non-technical skills are a key set of skills necessary for safe, effective patient management. The NOTSS (non-technical skills for surgeons) taxonomy defines the skills required of surgeons in the operating room, namely situation awareness, decision making, communication and teamwork, and leadership. Each of these skill categories pertains to a distinct skill set that needs to be practised and refined. Developing understanding of how we perform these skills and Identifying tools and techniques to integrate into daily practice allow us to both reduce human error and make our and our surrounding team performance more effective.     

Validation of laparoscopic surgical skills training outside the operating room: a long road

Background  Surgical skills training outside the operating room is beneficial. The best methods have yet to be identified. The authors aimed to document the predictive validity of simulation training in three different studies. Methods  Study 1 was a prospective, randomized, multicenter trial comparing performance in the operating room after training on a laparoscopic simulator and after no training. The Global Operative Assessment of Laparoscopic Skills (GOALS) was used to evaluate operative performance. Study 2 retrospectively reviewed the operative performance of junior residents before and after implementation of a laparoscopic skills training curriculum. Operative time was the variable used to determine resident improvement. Study 3 was a prospective, randomized trial evaluating intern operative performance of laparoscopic cholecystectomy in a porcine model before and after training on a simulator. Operative performance was assessed using GOALS. Results  All three studies failed to demonstrate predictive validity. With GOALS used as the assessment tool, no difference was found between trained and untrained residents in studies 1 and 3. In study 2, the trained group took significantly longer to complete a laparoscopic cholecystectomy than the untrained group. Conclusions  No correlation was found between the three types of training outside the operating room, and no improved operative performance was observed. Possible explanations include too few subjects, training introduced too late in the learning curve, and training criteria that were too easy. Additionally, simulator training focuses on precision, which may actually increase task time. Awareness of these issues can improve the design of future studies. This work was presented at the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) meeting, Philadelphia, PA, April 2008.     

Operative experience of general surgeons in a rural hospital

Background: There is a lack of published data on the range of procedures currently carried out by rural general surgeons. The aim of this study is to demonstrate the diversity of conditions encountered and managed by rural general surgeons, in order to determine the scope of training necessary to adequately equip trainees for rural practice. Methods: An audit of surgical procedures completed by two surgeons over 5 years from June 2004 to June 2009 in a rural Victorian hospital was carried out. Operations were further categorized into specialty groups. Results: A total of 8336 operations were completed over 5 years, by two surgeons servicing a catchment population of 54 000 people. Traditional general surgical procedures accounted for 44.3% with endoscopies accounting for a further 27.4% of cases. The remainder was accounted for by 11.8% orthopaedics, 4.5% urology, 4.4% vascular (including pacemaker insertion), 3.4% ear, nose and throat, 2.7% neurosurgery, 0.88% obstetrics and 0.42% thoracics. There was a need to be able to perform procedures well outside the field of traditional general surgery, including multisystem trauma, vascular emergencies, caesarean sections and post‐partum hysterectomy. Conclusion: General surgeons servicing small rural communities are required to carry out a variety of procedures outside the realms of traditional general surgery. Trainees need to acquire a broad skill base, and should be competent managing surgical emergencies across all specialties. A supportive team of medical, nursing and paramedical staff are required for achieving the best outcome for patients.     

When surgeons decide to become surgeons: new opportunities for surgical education

Background

When surgeons decide to become surgeons has important implications. If the decision is made prior to or early in medical school, surgical education can be more focused on surgical diseases and resident skills.

Methods

To determine when surgeons – compared with their nonsurgical colleagues – decide on their medical path, residents in surgery, internal medicine, obstetrics and gynecology, pediatrics, psychiatry, and emergency medicine were surveyed. Timing of residency choice, demographic data, personal goals, and reason for residency choice were queried.

Results

A total of 234 residents responded (53 surgical residents). Sixty-two percent of surgeons reported that they were “fairly certain” of surgery before medical school, 13% decided during their preclinical years, and 25% decided during their clerkship years. This compares with an aggregate 40%, 7%, and 54%, respectively, for the other 5 residency specialties. These differences were statistically significant (P = .001). When the 234 residents were asked about their primary motivation for choosing their field, 51% pointed to expected job satisfaction and 44% to intellectual curiosity, and only 3% mentioned lifestyle, prestige, or income.

Conclusions

General surgery residents decide on surgery earlier than residents in other programs. This may be advantageous, resulting in fast-tracking of these medical students in acquiring surgical knowledge, undertaking surgical research, and early identification for surgical residency programs. Surgical training in the era of the 80-hour work week could be enhanced if medical students bring much deeper knowledge of surgery to their first day of residency.     

The influence of practice schedules in the learning of a complex bone-plating surgical task

BACKGROUND: Practicing surgical tasks on bench models can be arranged in 3 ways: as the entire task, or as individual skills practiced separately in blocked or random order. The issue of the optimal practice schedule for the acquisition of surgical tasks is critical for enhancing training programs. METHOD: An orthopedic bone-plating task was practiced as a whole, or in parts in either a random or a blocked order. Learning was assessed on global ratings, checklists, and final product analysis before, immediately after, and an hour after practice. RESULTS: Checklists, and final product analysis, but not the global ratings showed that practicing the entire task resulted in the most learning, followed by the random practice. Practice of the skills in a blocked order yielded the least amount of learning. CONCLUSIONS: It is recommended that surgical tasks composed of several discrete skills should be practiced as a whole. However, if part practice is necessary, these skills should be arranged in random order to optimize learning.     

Effective instruction of vascular anastomosis in the surgical skills laboratory

BACKGROUND: A greater emphasis has recently been placed on laboratory-based training of core surgical skills and tasks, such as vascular anastomosis. Despite this emphasis, little objective data exist regarding the effectiveness of vascular anastomosis instruction in the laboratory setting. METHODS: Residents of all postgraduate levels received laboratory-based vascular anastomosis training. Each individual fashioned an end-to-side anastomosis between synthetic graft and porcine aorta before instruction (pretest). All subjects then received standardized anastomosis training with practice time. After training, the anastomosis was then repeated (posttest). Metrics included time to completion, anastomotic leakage, and visual characteristics of the completed product. RESULTS: Postteaching improvement was demonstrated by all residents (n = 32). Posthoc analysis showed statistically significant improvements in junior residents only. CONCLUSIONS: Laboratory-based training can effectively improve the ability of residents to perform vascular anastomosis immediately after training. Junior residents may gain a greater advantage from laboratory teaching.     

Assessment of surgical skills of trainees in the UK

Surgical training and assessment in the UK has been criticised in the past for lacking transparency, reliability and validity. The new Intercollegiate Surgical Curriculum Programme (ISCP) has a well-defined, competence-based syllabus and a system of workplace-based assessments and examinations that map to the syllabus. The main aims of workplace-based assessment are to aid learning through objective feedback and to provide evidence that the competencies required to progress to the next level of training have been achieved. Reduction in surgical experience means that more training will need to be undertaken on simulations, although experience and assessment in the operating room must remains the 'gold-standard'. Simulation training will require the provision of properly resourced surgical skills facilities in every hospital. The key to reliable assessment and constructive feedback is well-trained trainers. Training is a skill that must be learned, and assessment and feedback techniques form part of this. In surgery, it has been assumed that all consultants are trainers but this is clearly not the case. Surgeons will need to follow the example of primary care, where trainers are selected from experienced general practitioners who demonstrate enthusiasm and ability. The reward for the trainer should be protected time for training. The reward for the National Health Service will be better trained surgeons.     

Low-fidelity exercises for basic surgical skills training and assessment

Background

The goal of this study was to develop and validate low-fidelity exercises for basic surgical skills training and assessment.

Methods

Five low-fidelity exercises were developed and administered to 40 participants. Participants were classified as novice or proficient based on level of training. Objective metrics were used for scoring. The cost for assembling 1 complete set of 5 exercises and all necessary supplies for practice and evaluation was $150. Once this set was assembled, the subsequent cost for replacement material was $5/participant examined.

Results

Twenty-four participants were categorized as novice and 16 as proficient. Proficient participants scored significantly higher than novice participants (P <.05) for exercises assessing needle-driving skills, 2-hand coordination, and knot tying, thus establishing construct validity. Cronbach's alpha coefficient for internal consistency was .78, which demonstrates the exercises' reliability as a testing instrument.

Conclusions

These data provide preliminary evidence of construct validity and internal consistency for a cost-effective series of low-fidelity basic surgical skills exercises.     

Laying the foundation of surgical skills for trainees (Residents)

The role of the surgical skills laboratory is becoming increasingly important as surgical service needs evolve and educational processes seek to become more robust. The skills laboratory offers an opportunity to better prepare residents for their clinical rotations and to accelerate their clinical effectiveness. The Royal Australasian College of Surgeons has developed and introduced a new standardised basic surgical skills curriculum that can be delivered sustainably in centres throughout Australia and New Zealand. A technical skill set required by all surgical disciplines was compiled, and each skill was deconstructed into cognitive steps. Models were devised to simulate an application of each skill, and standardized demonstrations were created and recorded to ensure uniform teaching. Extensive resource material supports the course programme, with information for tutors, technicians, course directors and participants. Currently, clinical rotations offer inconsistent opportunities for trainees to practise their skills after the course. Recruitment and retention of tutors remains a key challenge.     

Analysis of hand motion differentiates expert and novice surgeons

Background

The number of operations performed by a surgeon may be an indicator of surgical skill. The hand motions made by a surgeon also reflect skill and level of expertise. We hypothesized that the hand motions of expert and novice surgeons differ significantly, regardless of whether they are familiar with specific tasks during an operation.

Methods

This study compared 11 expert surgeons, each of whom had performed >100 laparoscopic procedures, and 27 young surgeons, each of whom had performed <15 laparoscopic procedures. Each examinee performed a specific skill assessment task, in which instrument motion was monitored using magnetic tracking system. We analyzed the paths of the centers of gravity of the tips of the needle holders and the relative paths of the tips using two mathematical methods of detrended fluctuation analysis and unstable periodic orbit analysis.

Results

Detrended fluctuation analysis showed that the exponent in the function describing the initial scaling exponent (α₁) differed significantly for experts and novices, being close to 1.0 and 1.5, respectively (P < 0.01). This indicated that the expert group had a greater long-range coherence with an intrinsic sequence and smooth continuity among a series of motions. Likewise, unstable periodic orbit analysis showed that the second period of unstable orbit was significantly longer for experts in comparison with novices (P < 0.01). This demonstrates mathematically that the hands of experts are more stable when performing laparoscopic procedures.

Conclusions

Objective evaluation of hand motion during a simulated laparoscopic procedure showed a significant difference between experts and novices.