Assessment of technical and nontechnical skills in surgical residents

Background

Surgical competence encompasses both technical and nontechnical skills. This study seeks to evaluate the validity evidence for a comprehensive surgical skills examination and to examine the relationship between technical and nontechnical skills.

Methods

Six examination stations assessing both technical and nontechnical skills, conducted yearly for surgical trainees (n = 120) between 2010 and 2014 are included.

Results

The assessment tools demonstrated acceptable internal consistency. Interstation reliability for technical skills was low (alpha = .39). Interstation reliability for the nontechnical skills was lower (alpha range −.05 to .31). Nontechnical skills domains were strongly correlated, ranging from r = .65, P < .001 to .86, P < .001. The associations between nontechnical and technical skills were inconsistent, ranging from poor (r = −.06; P = .54) to moderate (r = .45; P < .001).

Conclusions

Multiple samplings of integrated technical and nontechnical skills are necessary to assess overall surgical competency.     

Students’ expectations on the surgery clerkship exceed those of residents and faculty

Purpose

The perceptions and expectations of students on the surgery clerkship were evaluated and compared with those of surgical residents and faculty.

Methods

A voluntary and anonymous survey was distributed to third year medical students, general surgery residents, and surgery faculty. Statistical analysis was performed using the χ² and Cochran-Mantel-Haenszel tests (P value <0.05 was significant).

Results

Medical students, surgery residents and faculty largely agreed on student responsibilities during morning rounds. However, more students (96.9%) than residents (86.2%) and faculty (77.3%) believed they should be writing progress notes, and fewer students (85%) than residents (100%) and faculty (95.5%) thought they should be taught during morning rounds (P < 0.01 and P = 0.04, respectively). The expected and actual educational experience on the surgical clerkship was similar. The amount of instruction provided each week by residents and faculty was similar to the amount expected by students; each group believed that surgical residents were good educators. Students, residents, and faculty agreed on many essential skills for a student to learn by the clerkship’s end; however, more students (85.8%) than residents (58.6%) and faculty (68.2%) viewed identifying surgical complications as an essential skill (P < 0.01).

Conclusions

Overall, the expectations of the medical students, surgical residents, and surgical faculty on the surgery clerkship were well matched. However, there were some instances where the students thought they should be more involved in patient care than did the residents and the faculty. It is important for surgical educators to be aware of these discrepancies to fully maximize the educational experience of medical students.     

American College of Surgeons/Association for Surgical Education medical student simulation-based surgical skills curriculum needs assessment

Background

Simulation can enhance learning effectiveness, efficiency, and patient safety and is engaging for learners.

Methods

A survey was conducted of surgical clerkship directors nationally and medical students at 5 medical schools to rank and stratify simulation-based educational topics. Students applying to surgery were compared with others using Wilcoxon's rank-sum tests.

Results

Seventy-three of 163 clerkship directors (45%) and 231 of 872 students (26.5%) completed the survey. Of students, 28.6% were applying for surgical residency training. Clerkship directors and students generally agreed on the importance and timing of specific educational topics. Clerkship directors tended to rank basic skills, such as examination skills, higher than medical students. Students ranked procedural skills, such as lumbar puncture, more highly than clerkship directors.

Conclusions

Surgery clerkship directors and 4th-year medical students agree substantially about the content of a simulation-based curriculum, although 4th-year medical students recommended that some topics be taught earlier than the clerkship directors recommended. Students planning to apply to surgical residencies did not differ significantly in their scoring from students pursuing nonsurgical specialties.     

The use of a lightly preserved cadaver and full thickness pig skin to teach technical skills on the surgery clerkship—a response to the economic pressures facing academic medicine today

Background

In response to declining instruction in technical skills, the authors instituted a novel method to teach basic procedural skills to medical students beginning the surgery clerkship.

Methods

Sixty-three medical students participated in a skills training laboratory. The first part of the laboratory taught basic suturing skills, and the second involved a cadaver with pig skin grafted to different anatomic locations. Clinical scenarios were simulated, and students performed essential procedural skills.

Results

Students learned most of their suturing skills in the laboratory skills sessions, compared with the emergency room or the operating room (P = .01). Students reported that the laboratory allowed them greater opportunity to participate in the emergency room and operating room. Students also felt that the suture laboratory contributed greatly to their skills in wound closure. Finally, 90% of students had never received instruction on suturing, and only 12% had performed any procedural skills before beginning the surgery rotation.

Conclusions

The laboratory described is an effective way of insuring that necessary technical skills are imparted during the surgery rotation.     

Assessment of surgical residents in a vascular anastomosis laboratory

Background

We designed a simple, low-cost workshop to teach surgical residents the basic skills of vascular anastomosis. We studied our ability to identify objective procedural and end-product metrics that could be used to measure improvement in vascular anastomotic skill before and after training.

Materials and methods

Ten postgraduate year 2 residents without previous vascular surgery experience and four attending surgeons (expert) performed end-to-side anastomosis using a synthetic graft. The residents were taught the basic skills of vascular anastomosis during three didactic workshops. The objective metrics included volume leakage after saline perfusion (leak) and the time needed to complete the anastomosis. Penalty points were assigned for broken sutures, air knots, locking sutures, and failure to maintain an outside-in to inside-out technique. The leak, time, and penalties before and after training were compared.

Results

The mean leak was 70.4 ± 13.7 mL and the mean completion time was 18.7 ± 3 min for the pretraining group versus 45.3 ± 10.6 mL (P < 0.01) and 8.5 ± 1 min (P < 0.001), respectively, for the attending group. After training, significant improvement was seen in resident leak (46.7 ± 6.8 mL; P < 0.001) and completion time (14.4 ± 3 min; P < 0.01). Leak was similar between the post-training and expert groups (46.7 ± 6.8 mL and 45.3 ± 10.6 mL, respectively; P = 0.77); however, a significant difference for the completion time remained (14.4 ± 3.0 min and 8.5 ± 1 min, respectively; P < 0.01). The mean number of technical errors improved from 2.7 in the pretraining group to zero for the post-training group after completing the workshop.

Conclusions

We have reported an easy to implement workshop for teaching surgical residents the basic skills of performing vascular anastomosis.     

Influence of medical students' past experiences and innate dexterity on suturing performance

Background

Medical students often site their ability to excel at technical tasks as justification for choosing surgery as a career path. We sought to investigate how medical students' dexterity skills and past experiences correlated with suturing performance.

Methods

Sixty-four 3rd-year medical students were surveyed about previous experiences that involved manual dexterity. Technical skills were then measured using a validated test of manual dexterity and subcuticular closure of a pig's foot incision. Spearman's rank correlation coefficients determined correlation between variables.

Results

Previous experiences, self-assessment of dexterity, prior suturing, and current interest in surgery did not significantly correlate with manual dexterity or suturing skill scores. Innate manual dexterity score was the only significant correlating factor to suture skill score (Spearman's rank correlation coefficient = .336; P = .007).

Conclusions

Innate manual dexterity skills are predictive of initial surgical suturing performance regardless of past student experiences. Interventions aimed at improving early surgical technique should be optimally focused on dexterity training.     

Palliative and end-of-life care training during the surgical clerkship

Background

In 2000, the Liaison Committee on Medical Education required that all medical schools provide experiential training in end-of-life care. To adhere to this mandate and advance the professional development of medical students, experiential training in communication skills at the end-of-life was introduced into the third-year surgical clerkship curriculum at Wright State University Boonshoft School of Medicine.

Materials and methods

In the 2007–08 academic year, 97 third-year medical students completed six standardized end-of-life care patient scenarios commonly encountered during the third-year surgical clerkship. Goals and objectives were outlined for each scenario, and attending surgeons graded student performances and provided formative feedback.

Results

All 97 students, 57.7% female and average age 25.6 ± 2.04 y, had passing scores on the scenarios: (1) Adult Hospice, (2) Pediatric Hospice, (3) Do Not Resuscitate, (4) Dyspnea Management/Informed Consent, (5) Treatment Goals and Prognosis, and (6) Family Conference. Scenario scores did not differ by gender or age, but students completing the clerkship in the first half of the year scored higher on total score for the six scenarios (92.8% ± 4.8% versus 90.5% ± 5.0%, P = 0.024).

Conclusions

Early training in end-of-life communication is feasible during the surgical clerkship in the third-year of medical school. Of all the scenarios, “Conducting a Family Conference” proved to be the most challenging.     

A mentorship-based preclinical elective increases exposure,confidence, and interest in surgery

Background

The predicted shortage of surgeons is of growing concern with declining medical student interest in surgical careers. We hypothesized that earlier exposure to operative experiences and the establishment of resident mentors through a preclinical elective would enhance student confidence and interest in surgery.

Methods

We developed a preclinical elective in surgery, which served as an organized curriculum for junior medical students to experience surgery through a paired resident-mentorship model. We assessed student exposure and confidence with clinical activities before and after the elective (N = 24, 100% response rate). We compared these students with a cohort of peers not enrolled in the elective (N = 147, 67% response rate).

Results

We found significantly improved confidence (2.8 vs 4.4) and clinical exposure (2.4 vs 4.3) before versus after the elective, with precourse scores equal to matched peers.

Conclusions

This elective incorporates elements that have been shown to positively influence student decision making in surgical career choice. The mentorship model promotes residents as educators, whereas the elective provides a means for early identification of students interested in surgery.     

Standardized and quality-assured video-recorded examination in undergraduate education: informed consent prior to surgery

Background

Communication skills combined with specialized knowledge are fundamental to the doctor–patient relationship in surgery. During a single-station video-recorded objective structured clinical examination (VOSCE), students were tasked with obtaining informed consent. Our aim was to develop a standardized and quality-assured assessment method in undergraduate education.

Methods

One hundred fifty-five students in their fifth year of medical school (78 videos) participated in a summative VOSCE within the framework of the teaching module “Operative Medicine.” They prepared for three clinical scenarios and the surgical procedures involved. The examination comprised participants having to obtain informed consent from simulated patients, video recording their performance. Students were assessed by two independent raters, the background of one of whom was nonsurgical. Results were statistically tested using SPSS.

Results

Students' scores were all beyond the pass mark of 70%, averaging 91.0% (±4.0%), 88.4% (±4.4%), and 87.0% (±4.7%) for the appendectomy, cholecystectomy, and inguinal hernia repair checklist, respectively. Most items (68%–89% of the checklists) were found to have fair to excellent discrimination values. Cronbach's α values ranged between 0.565 and 0.605 for the individual checklists. Interrater agreement was strong (Pearson correlation coefficient = 0.80, P < 0.01; intraclass correlation coefficient 2.1 = 0.78).

Conclusions

The VOSCE is both feasible and reliable as a method of assessing student communication skills and the application of clinical knowledge while obtaining informed consent in surgery. This method is efficient (flexible rating outside normal working hours possible with reductions in administrative load) and may be used for high-stakes evaluation of student performance.     

Lights, camera, surgery: a novel pilot project to engage medical students in the development of pediatric surgical learning resources

Background/Purpose

It is important to engage junior medical students in the pediatric surgical environment to showcase possible future career choices. Our aim was to assess how the students valued their experience in the realm of clinical learning, exposure to surgical careers, and development of skill sets necessary for creating learning resources.

Methods

A novel pilot project entitled Lights, Camera, Surgery engaged 13 medical students in the production of instructional videos of basic surgical procedures. An electronic survey questionnaire allowed the students to provide formal feedback on the project outcomes.

Results

Of the students who responded, 87.5% expressed appreciation of the enhanced clinical experience. All of the students either agreed or strongly agreed that the project afforded them valuable leadership experience, practical skills in creating educational learning resources, and opportunities to explore careers in surgery. All of the students either agreed or strongly agreed that the project allowed them to gain valuable skills in educational video production. The project videos are now available as educational tools.

Conclusions

Engaging medical students in the production of surgical videos potentially improves leadership skills and promotes the use of educational resources while affording them opportunities to explore pediatric surgery as a future career choice.     

Do laparoscopic skills transfer to robotic surgery?

Background

Identifying the set of skills that can transfer from laparoscopic to robotic surgery is an important consideration in designing optimal training curricula. We tested the degree to which laparoscopic skills transfer to a robotic platform.

Methods

Fourteen medical students and 14 surgery residents with no previous robotic but varying degrees of laparoscopic experience were studied. Three fundamentals of laparoscopic surgery tasks were used on the laparoscopic box trainer and then the da Vinci robot: peg transfer (PT), circle cutting (CC), and intracorporeal suturing (IS). A questionnaire was administered for assessing subjects' comfort level with each task.

Results

Standard fundamentals of laparoscopic surgery scoring metric were used and higher scores indicate a superior performance. For the group, PT and CC scores were similar between robotic and laparoscopic modalities (90 versus 90 and 52 versus 47; P > 0.05). However, for the advanced IS task, robotic-IS scores were significantly higher than laparoscopic-IS (80 versus 53; P < 0.001). Subgroup analysis of senior residents revealed a lower robotic-PT score when compared with laparoscopic-PT (92 versus 105; P < 0.05). Scores for CC and IS were similar in this subgroup (64 ± 9 versus 69 ± 15 and 95 ± 3 versus 92 ± 10; P > 0.05). The robot was favored over laparoscopy for all drills (PT, 66.7%; CC, 88.9%; IS, 94.4%).

Conclusions

For simple tasks, participants with preexisting skills perform worse with the robot. However, with increasing task difficulty, robotic performance is equal or better than laparoscopy. Laparoscopic skills appear to readily transfer to a robotic platform, and difficult tasks such as IS are actually enhanced, even in subjects naive to the technology.     

Correlating surgical clerkship evaluations with performance on the National Board of Medical Examiners examination

Background

Evaluation of medical students during the surgical clerkship is controversial. Performance is often based on subjective scoring, whereas objective knowledge is based on written examinations. Whether these measures correspond or are relevant to assess student performance is unknown. We hypothesized that student evaluations correlate with performance on the National Board Of Medical Examiners (NBME) examination.

Methods

Data were collected from the 2011–2012 academic year. Medical students underwent a ward evaluation using a seven-point Likert scale assessing six educational competencies. Students also undertook the NBME examination, where performance was recorded as a percentile score adjusted to national standards.

Results

A total of 129 medical students were studied. Scores on the NBME ranged from the 52nd to the 96th percentile with an average in the 75th percentile (±9). Clerkship scores ranged from 3.2–7.0 with a mean of 5.7 (±0.8). There was a strong positive association between higher NBME scores and higher clerkship evaluations shown by a Pearson correlation coefficient of 0.47 (P < 0.001). Students clustered with below average ward evaluations (3.0–4.0) were in the 69.5th percentile of NBME scores, whereas students clustered with above average ward evaluations (6.0–7.0) were in the 79.2th percentile (P < 0.001).

Conclusions

A strong positive relationship exists between subjective ward evaluations and NBME performance. These data may afford some confidence to surgical faculty and surgical resident ability to accurately evaluate medical students during clinical clerkships. Understanding factors in student performance may help in improving the surgical clerkship experience.     

Accuracy and content of medical student midclerkship self-evaluations

Background

Midclerkship self-evaluations (MCSEs) require students to reflect on their knowledge, skills, and behaviors. We hypothesized that MCSEs would be consistent with supervisor midpoint evaluations during a surgical clerkship.

Methods

MCSEs of 153 students who completed our surgery clerkship in 2 academic years were compared with supervisor midclerkship evaluations. The quantitative domains of the MCSE and supervisor evaluation were compared for accuracy. Identified areas of strengths and weakness were evaluated for thematic consistency.

Results

Student MCSE scoring was accurate across evaluated domains most of the time; when students were inaccurate, they tended to underrate themselves. Students and supervisors most often identified cognitive skills as areas for improvement and noncognitive skills predominated as student strengths.

Conclusions

Medical students can accurately identify their strengths and weaknesses in the context of an MCSE. Based on these findings, knowledge acquisition and application by medical students in the clinical setting should be emphasized in undergraduate medical education.     

Duty hours and perceived competence in surgery: are interns ready?

Background

A fundamental shift in the structure of many surgical training programs has occurred after the July 2011 rule changes. Our intern didactic program was intensified in 2011 with targeted lectures, laboratories, and clinical cases as well as direct supervision until competency was achieved for basic clinical problems. We sought to compare interns' perceived preparedness throughout and at the end of the academic years before and after July 2011.

Materials and methods

Intern perceptions of preparedness to manage common clinical scenarios and perform procedures in general surgery were serially surveyed in academic years ending in 2011 and 2012 based on the Residency Review Committee supervision guidelines.

Results

Interns felt less prepared across all measured domains from 2011–2012. Interns felt significantly less prepared to manage hypotension (3.00/4 points to 2.67/4 points; P = 0.04), place a tube thoracostomy (2.45/4 points to 1.92/4 points; P = 0.04), or perform an inguinal hernia repair (1.91/4 points to 0.92/4 points; P = 0.01) without supervision. Interns were also significantly less likely to agree that they were able to gain clinical skills based on experience (4.31/5 points versus 4.15/5 points; P = 0.02). Longitudinal analysis throughout internship demonstrated improved preparedness to manage common clinical problems and perform procedures between the second and the fifth months of internship.

Conclusions

First-year residents after July 2011 felt less prepared in the topics surveyed than those before July 2011. Interns made the greatest gains in preparedness between months 2 and 5, suggesting that despite planned interventions, no substitute currently exists for actual clinical experience. Planned educational interventions to improve intern preparedness are also indicated.     

International surgical clerkship rotation: perceptions and academic performance

Background

Concerns about international training experiences in medical school curricula include the effect on student learning. We studied the educational effect of an international elective integrated into a traditional third-year (M3) surgical clerkship.

Methods

A 1-week surgical elective in Haiti was available to M3 students during the conventional 8-week surgical clerkship each year for the 4 academic years 2008 to 2011. The authors collected student and surgeon perceptions of the elective using a mixed-methods web-based survey. Statistical analysis compared the academic performance of participating M3s relative to nonparticipating peers.

Results

Twenty-eight (100%) students (41 trip weeks) and 3 (75%) surgeons responded. Twenty-five (89%) students believed the elective provided appropriate clinical training. Surgeon responses were consistent with students' reported perceptions.Strengths included unique clinical experiences and close interactions with faculty. Criticisms included recurring overwhelming clinical responsibilities and lack of local provider involvement.Academic performance of participants versus nonparticipants in the same clerkship term were statistically insignificant.

Conclusions

This study demonstrates the feasibility of integrating global health experiences into traditional medical student clinical curricula. The effects on less tangible attributes such as leadership skills, fostering teamwork, and cultural competency require future investigation.     

Is there a place for medical students as teachers in the education of junior residents?

Background

The aim of this study was to investigate a novel resident education model that turns the traditional surgical hierarchy upside down, termed a “reverse” peer-assisted learning curriculum.

Methods

Thirty surgical topics were randomized between medical students and chief residents on each clinical team, with 1 topic being presented briefly during morning rounds. An exam evaluating junior residents' knowledge of these topics was administered before and after 1 month of presentations. A questionnaire was distributed to evaluate the junior residents' perceptions of this teaching model.

Results

Thirty-four residents participated. There was a significant improvement in the mean examination score (54 vs 74, P < .05). No significant difference was noted in the mean score differentials of topics presented by either the medical students or the chief resident (21 vs 18, P = .22). More than 80% of the residents responded positively about the effectiveness of this exercise and agreed that they would like to see this model used on other services.

Conclusions

This study confirms the hypothesis that medical students can teach surgical topics to junior residents at least as effectively as their chief residents.     

A focused curriculum in surgical oncology for the third-year medical students

Background

Educating medical students in surgical subspecialty fields can be challenging, and the optimal timing and curriculum remain unknown. Despite advocacy for earlier exposure, competing core clerkship rotations often leave little time for subspecialty fields. We report our experience with a novel, short, and focused curriculum in surgical oncology for the third-year medical students.

Methods

A 2-wk (2009–2010) and a 4-wk (2010–2011) curriculum in surgical oncology were developed for the third-year students at a tertiary-referral cancer center, including formal didactics, rotation in clinical service of students' choosing (breast, gastrointestinal, endocrine, or melanoma), and case-based learning and presentation. Paired pre- and postrotation questionnaires were prospectively completed, including 20 items assessing knowledge and four items assessing experience. Grading was anonymous, and change in score was assessed by Wilcoxon signed-rank test.

Results

Paired questionnaires from 47 students (2-wk rotation, n = 26; 4-wk rotation, n = 21) showed a median improvement of three points (21.4%) from pre- to posttests (P < 0.001). The improvement did not differ by the length of rotation or by the specific clinical service. Nearly all (93%) reported a positive and inspiring experience. The most valuable avenue of learning was reported as the time spent with resident or fellow or attending (92%), followed by self-directed reading (62%) and didactic lectures (28%).

Conclusions

A short and focused curriculum in surgical oncology, including structured didactics and clinical rotation, had positive impact for the third-year students. Given the increasing work-hour limits, it is important to note that the time spent in the clinical setting continues to be ranked as the most educationally valuable by medical students.     

When surgeons decide to become surgeons: new opportunities for surgical education

Background

When surgeons decide to become surgeons has important implications. If the decision is made prior to or early in medical school, surgical education can be more focused on surgical diseases and resident skills.

Methods

To determine when surgeons – compared with their nonsurgical colleagues – decide on their medical path, residents in surgery, internal medicine, obstetrics and gynecology, pediatrics, psychiatry, and emergency medicine were surveyed. Timing of residency choice, demographic data, personal goals, and reason for residency choice were queried.

Results

A total of 234 residents responded (53 surgical residents). Sixty-two percent of surgeons reported that they were “fairly certain” of surgery before medical school, 13% decided during their preclinical years, and 25% decided during their clerkship years. This compares with an aggregate 40%, 7%, and 54%, respectively, for the other 5 residency specialties. These differences were statistically significant (P = .001). When the 234 residents were asked about their primary motivation for choosing their field, 51% pointed to expected job satisfaction and 44% to intellectual curiosity, and only 3% mentioned lifestyle, prestige, or income.

Conclusions

General surgery residents decide on surgery earlier than residents in other programs. This may be advantageous, resulting in fast-tracking of these medical students in acquiring surgical knowledge, undertaking surgical research, and early identification for surgical residency programs. Surgical training in the era of the 80-hour work week could be enhanced if medical students bring much deeper knowledge of surgery to their first day of residency.     

A framework for professionalism in surgery: what is important to medical students?

Background

The purposes of this study were to develop a comprehensive framework for professionalism in surgery and to determine which attributes are most valued by medical students.

Methods

A framework for professionalism in surgery, consisting of 11 attribute categories, was developed. All 3rd-year medical students (n = 168) participated in a focus group and completed a questionnaire regarding their perceptions about professionalism. Students' responses were transcribed verbatim, coded, and assigned attribute categories.

Results

Students rated respect as the most important attribute of professionalism (56%), followed by altruism (21%) and interpersonal skills (8%). Fifty-three percent of students witnessed unprofessional behavior among faculty members while on the surgical clerkship. Of these incidents, 74% were related to respect, 28% to practice improvement, and 1% to altruism.

Conclusions

Respect was rated as the single most important characteristic of professionalism and was the attribute with the most witnessed violations.