Is USMLE Step 1 score a valid predictor of success in surgical residency?

Background

Many programs rely extensively on United States Medical Licensing Examination (USMLE) scores for interviews/selection of surgical residents. However, their predictive ability remains controversial. We examined the association between USMLE scores and success in surgical residency.

Methods

We compared USMLE scores for 123 general surgical residents who trained in the past 20 years and their performance evaluation. Scores were normalized to the mean for the testing year and expressed as a ratio (1 = mean). Performances were evaluated by (1) rotation evaluations; (2) “dropouts;” (3) overall American Board of Surgery pass rate; (4) first-time American Board of Surgery pass rate; and (5) a retrospective comprehensive faculty evaluation. For the latter, 16 surgeons (average faculty tenure 22 years) rated residents on a 1 to 4 score (1 = fair; 4 = excellent).

Results

Rotation evaluations by faculty and “drop out” rates were not associated with USMLE score differences (dropouts had average above the mean). One hundred percent of general surgery practitioners achieved board certification regardless of USMLE score but trainees with an average above the mean had a higher first-time pass rate (P = .04). Data from the comprehensive faculty evaluations were conflicting: there was a moderate degree of correlation between board scores and faculty evaluations (r = .287, P = .001). However, a score above the mean was associated with a faculty ranking of 3 to 4 in only 51.7% of trainees.

Conclusion

Higher USMLE scores were associated with higher faculty evaluations and first-time board pass rates. However, their positive predictive value was only 50% for higher faculty evaluations and a high overall board pass rate can be achieved regardless of USMLE scores. USMLE Step 1 score is a valid tool for selecting residents but caution might be indicated in using it as a single selection factor.     

Increasing the number of integrated vascular surgery residency positions is important to address the impending shortage of vascular surgeons in the United States

Objective

The demand for vascular surgeons is expected to far exceed the current supply. In an attempt to decrease the training duration and to address the impending shortage, integrated vascular surgery residencies were approved and have expanded nationally. Meanwhile, vascular fellowships have continued to matriculate approximately 120 trainees annually. We sought to evaluate the supply and demand for integrated vascular residency positions as well as changes in the quality of applicants.

Do medical student's surgical examination scores correlate with performance markers?

Background

Some medical school training consists of oral examinations.

Methods

We conducted a 9-year review of third-year medical student examinations including oral examinations, National Board of Medical Examiners Surgery Subject Examination (SSE, ie, shelf), and United States Medical Licensing Examinations Step 1 and Step 2.

Results

Step 1 showed a moderate to strong association with Period 1 orals (Somers' D = .297, P < .001), but not Period 2 orals (Somers' D = .048, P = .053). Period 1 orals (percentage) had a strong association with SSE (Somers' D = .356, P < .001) and Step 2 (Somers' D = .368, P < .001). Period 2 orals (pass/fail) suggested a positive, but not statistically significant, association with SSE (Somers' D = .334, P = .085) and Step 2 (Somers' D = .370, P = .055). Step 1 shows a strong association with SSE (Somers' D = .490, P < .001). SSE showed a strong association with Step 2 (Somers' D = .506, P < .001).

Conclusions

Orals can be used to identify students who may have difficulty passing the SSE. Step 1 can be used to identify students at risk of poor performance on the SSE, and SSE can be used to identify students at risk for poor performance on Step 2.     

When surgeons decide to become surgeons: new opportunities for surgical education

Background

When surgeons decide to become surgeons has important implications. If the decision is made prior to or early in medical school, surgical education can be more focused on surgical diseases and resident skills.

Methods

To determine when surgeons – compared with their nonsurgical colleagues – decide on their medical path, residents in surgery, internal medicine, obstetrics and gynecology, pediatrics, psychiatry, and emergency medicine were surveyed. Timing of residency choice, demographic data, personal goals, and reason for residency choice were queried.

Results

A total of 234 residents responded (53 surgical residents). Sixty-two percent of surgeons reported that they were “fairly certain” of surgery before medical school, 13% decided during their preclinical years, and 25% decided during their clerkship years. This compares with an aggregate 40%, 7%, and 54%, respectively, for the other 5 residency specialties. These differences were statistically significant (P = .001). When the 234 residents were asked about their primary motivation for choosing their field, 51% pointed to expected job satisfaction and 44% to intellectual curiosity, and only 3% mentioned lifestyle, prestige, or income.

Conclusions

General surgery residents decide on surgery earlier than residents in other programs. This may be advantageous, resulting in fast-tracking of these medical students in acquiring surgical knowledge, undertaking surgical research, and early identification for surgical residency programs. Surgical training in the era of the 80-hour work week could be enhanced if medical students bring much deeper knowledge of surgery to their first day of residency.     

American Board of Surgery examinations: can we identify surgery residency applicants and residents who will pass the examinations on the first attempt?

Background

The Residency Review Committee requires that 65% of general surgery residents pass the American Board of Surgery qualifying and certifying examinations on the first attempt. The aim of this study was to identify predictors of successful first-attempt completion of the examinations.

Methods

Age, sex, Alpha Omega Alpha Honor Medical Society status, class rank, honors in third-year surgery clerkship, interview score, rank list number, National Board of Medical Examiners/United States Medical Licensing Examination scores, American Board of Surgery In-Training Examination scores, resident awards, and faculty evaluations of senior residents were reviewed. Graduates who passed both examinations on the first attempt were compared with those who failed either examination on the first attempt.

Results

No subjective evaluations of performance predicted success other than resident awards. Significant objective predictors of successful first-attempt completion of the examinations were Alpha Omega Alpha status, ranking within the top one third of one's medical student class, National Board of Medical Examiners/United States Medical Licensing Examination Step 1 (>200, top 50%) and Step 2 (>186.5, top 3 quartiles) scores, and American Board of Surgery In-Training Examination scores >50th percentile (postgraduate years 1 and 3) and >33rd percentile (postgraduate years 4 and 5).

Conclusions

Residency programs can use this information in selecting residents and in identifying residents who may need remediation.     

How Should Unmatched Orthopaedic Surgery Applicants Proceed?

Background

Obtaining an orthopaedic surgery residency is competitive. Advisors must understand what factors may help unmatched candidates reapply successfully.

Questions/purposes

We determined (1) the attitude of leaders of orthopaedic surgery residency programs toward interviewing unmatched students; (2) whether a surgical internship or a research year is preferred in considering reapplicants; (3) the importance of United States Medical Licensing Examination (USMLE) scores, recommendations, and Alpha Omega Alpha (AOA) membership; and (4) whether academic and nonacademic programs evaluate reapplicants differently.

Methods

We sent an anonymous 19-question survey to 151 Accreditation Council for Graduate Medical Education (ACGME)-accredited orthopaedic surgery residency programs in five waves, 1 week apart (December 5, 2009–January 5, 2010). Investigators were blinded to the respondents’ identities.

Results

Ninety-one of the 151 programs (60%) responded. Sixty-eight of the 91 programs (75%) stated they rarely accept unmatched applicants. Sixty-eight programs (75%) agreed an unmatched applicant should do a surgery internship for 1 year. Of the 36 programs that recommended a research year, 32 were academic programs. Academic programs were more likely than nonacademic programs to view as important new recommendations (85% versus 67%), minimum scores of 220 on Step I (67% versus 49%) and Step II (64% versus 36%), and AOA membership (85% versus 67%).

Conclusions

By completing a surgical internship, unmatched students may increase their chances of matching. Students considering academic programs should ensure their academic record meets certain benchmarks and may consider a research year but risk limiting their acceptance to academic programs.     

The importance of residency program size and location on American Osteopathic Board of Surgery In-Training Examination outcomes

Background

The American Osteopathic Board of Surgery In-Training Examination (AOBSITE) is administered to general surgery residency training programs. Based on findings in allopathic training, we hypothesize that larger programs will outperform smaller programs and that Southern programs will perform lower than other geographic regions.

Materials and Methods

In this retrospective study, the performance on the AOBSITE was obtained for all of the osteopathic general surgery programs from 2008 to 2012. To test if program size was related to AOBSITE performance, simple linear regression was performed. Geographic differences in median performance between states and US Census Bureau regions were evaluated using Kruskal–Wallis tests. Nonparametric statistics were performed using an α = 0.05.

Results

From 2008 to 2012, there were 49 general surgery residency training programs and 2278 examinees evaluated. The median raw performance by general surgery residency training program was 168.0 (IQR [161.8–177.7]). The weighted median standardized performance by general surgery residency training program was 487.8 (IQR [462.8–528.0]). Simple linear regression analyses showed that the slope of the least-square regression line was greater than zero for raw performance (P = 0.048) and standardized performance (P = 0.005). A Kruskal–Wallis test showed that there were no differences in raw performance or standardized performance by US Census Bureau Region or by state (all P > 0.05).

Conclusions

Overall, larger general surgery residency training programs outperform smaller programs on the AOBSITE and that there are no geographical differences in performance by state or region.     

Operative confidence of graduating surgery residents: a training challenge in a changing environment

Background

Given the recent changes in general surgical training, this study was undertaken to assess the confidence of graduating general surgery residents in performing open operations and to determine factors that are associated with increased confidence.

Methods

A survey was sent to the 5th-year general surgery residents in the Northeast. Respondents were queried regarding demographics, program characteristics and asked to rate their confidence in performing open operations. We compared those who indicated confidence with those who did not.

Results

We received 232 responses: 74% male, 70% from university programs, and 50% from programs affiliated with a Veterans Affairs Hospital. Fifty-two percent expressed confidence in their ability to practice independently after residency. Operative confidence varied with sex, type of training program, affiliation to a Veterans Affairs Hospital, and surgical volume.

Conclusions

Graduating surgical residents indicated a significant lack of confidence in performing a variety of open surgical procedures. Analyzing and addressing this confidence deficit merits further study.     

Pregnancy among residents enrolled in general surgery: a nationwide survey of attitudes and experiences

Background

Medical student interest in general surgery has declined, and the lack of adequate accommodation for pregnancy and parenting during residency training may be a deterrent. We explored resident and program director experiences with these issues in general surgery programs across Canada.

Methods

Using a web-based tool, residents and program directors from 16 Canadian general surgery programs were surveyed regarding their attitudes toward and experiences with pregnancy during residency.

Results

One hundred seventy-six of 600 residents and 8 of 16 program directors completed the survey (30% and 50% response rate, respectively). Multiple issues pertaining to pregnancy during surgical residency were reported including the lack of adequate policies for maternity/parenting, the major obstacles to breast-feeding, and the increased workload for fellow resident colleagues. All program directors reported the lack of a program-specific maternity/parenting policy.

Conclusions

General surgery programs lack program-specific maternity/parenting policies. Several issues have been highlighted in this study emphasizing the importance of creating and implementing such a policy.     

Innovation in internship preparation: an operative anatomy course increases senior medical students' knowledge and confidence

Background

An operative anatomy course was developed within the construct of a surgical internship preparatory curriculum. This course provided fourth-year medical students matching into a surgical residency the opportunity to perform intern-level procedures on cadavers under the guidance of surgical faculty members.

Methods

Senior medical students performed intern-level procedures on cadavers with the assistance of faculty surgeons. Students' confidence, anxiety, and procedural knowledge were evaluated both preoperatively and postoperatively. Preoperative and postoperative data were compared both collectively and based on individual procedures.

Results

Student confidence and procedural knowledge significantly increased and anxiety significantly decreased when preoperative and postoperative data were compared (P < .05). Students reported moderate to significant improvement in their ability to perform a variety of surgical tasks.

Conclusions

The consistent improvement in confidence, knowledge, and anxiety justifies further development of an operative anatomy course, with future assessment of the impact on performance in surgical residency.     

Influence of medical students' past experiences and innate dexterity on suturing performance

Background

Medical students often site their ability to excel at technical tasks as justification for choosing surgery as a career path. We sought to investigate how medical students' dexterity skills and past experiences correlated with suturing performance.

Methods

Sixty-four 3rd-year medical students were surveyed about previous experiences that involved manual dexterity. Technical skills were then measured using a validated test of manual dexterity and subcuticular closure of a pig's foot incision. Spearman's rank correlation coefficients determined correlation between variables.

Results

Previous experiences, self-assessment of dexterity, prior suturing, and current interest in surgery did not significantly correlate with manual dexterity or suturing skill scores. Innate manual dexterity score was the only significant correlating factor to suture skill score (Spearman's rank correlation coefficient = .336; P = .007).

Conclusions

Innate manual dexterity skills are predictive of initial surgical suturing performance regardless of past student experiences. Interventions aimed at improving early surgical technique should be optimally focused on dexterity training.     

Students’ expectations on the surgery clerkship exceed those of residents and faculty

Purpose

The perceptions and expectations of students on the surgery clerkship were evaluated and compared with those of surgical residents and faculty.

Methods

A voluntary and anonymous survey was distributed to third year medical students, general surgery residents, and surgery faculty. Statistical analysis was performed using the χ² and Cochran-Mantel-Haenszel tests (P value <0.05 was significant).

Results

Medical students, surgery residents and faculty largely agreed on student responsibilities during morning rounds. However, more students (96.9%) than residents (86.2%) and faculty (77.3%) believed they should be writing progress notes, and fewer students (85%) than residents (100%) and faculty (95.5%) thought they should be taught during morning rounds (P < 0.01 and P = 0.04, respectively). The expected and actual educational experience on the surgical clerkship was similar. The amount of instruction provided each week by residents and faculty was similar to the amount expected by students; each group believed that surgical residents were good educators. Students, residents, and faculty agreed on many essential skills for a student to learn by the clerkship’s end; however, more students (85.8%) than residents (58.6%) and faculty (68.2%) viewed identifying surgical complications as an essential skill (P < 0.01).

Conclusions

Overall, the expectations of the medical students, surgical residents, and surgical faculty on the surgery clerkship were well matched. However, there were some instances where the students thought they should be more involved in patient care than did the residents and the faculty. It is important for surgical educators to be aware of these discrepancies to fully maximize the educational experience of medical students.     

The Advanced Trauma Operative Management course in a Canadian residency program

Background

The Advanced Trauma Operative Management (ATOM) course was first introduced into Canada in 2003 at the University of Toronto, with senior general surgery residents being the primary focus. We present an assessment of the course in this Canadian general surgery residency program.

Methods

We compared trainees'' pre-and postcourse self-efficacy scores and multiple choice question (MCQ) examination results, using paired t tests and resident (n = 24) and faculty (n = 7) course ratings made according to a 10-item, 5-point Likert scale. Faculty were previously trained as ATOM instructors.

Results

Mean pre-and postcourse self-efficacy scores were 68.9 (standard deviation [SD] 24.0) and 101.4 (SD 14.8), respectively (p < 0.001). Mean pre-and post-MCQ scores were 16.4 (SD 3.2) and 18.8 (SD 2.7), respectively (p = 0.006). On the Likert scale (1 = strongly disagree, 5 = strongly agree), all faculty and residents rated the following items as 4–5: objectives were met; knowledge, skills, clinical training, judgment and confidence improved; the live animal is a useful representation of clinical trauma; and the course should be continued but would be more appropriate for the fourth rather than the fifth year of residency. Residents rated as 1–2 the item that the human cadaver would be preferable for learning the surgical skills. Of 24 residents, 20 rated as 3 or less the item stating that the course prepares them for trauma management more adequately than their regular training program.

Conclusion

Self-efficacy, trauma knowledge and skills improved significantly with ATOM training. Preference was expressed for the live animal versus cadaver model, for ATOM training in the fourth rather than fifth year of residency and for the view that it complements general surgery trauma training. The data suggest that including ATOM training in Canadian general surgical residency should be considered.     

American College of Surgeons/Association for Surgical Education medical student simulation-based surgical skills curriculum needs assessment

Background

Simulation can enhance learning effectiveness, efficiency, and patient safety and is engaging for learners.

Methods

A survey was conducted of surgical clerkship directors nationally and medical students at 5 medical schools to rank and stratify simulation-based educational topics. Students applying to surgery were compared with others using Wilcoxon's rank-sum tests.

Results

Seventy-three of 163 clerkship directors (45%) and 231 of 872 students (26.5%) completed the survey. Of students, 28.6% were applying for surgical residency training. Clerkship directors and students generally agreed on the importance and timing of specific educational topics. Clerkship directors tended to rank basic skills, such as examination skills, higher than medical students. Students ranked procedural skills, such as lumbar puncture, more highly than clerkship directors.

Conclusions

Surgery clerkship directors and 4th-year medical students agree substantially about the content of a simulation-based curriculum, although 4th-year medical students recommended that some topics be taught earlier than the clerkship directors recommended. Students planning to apply to surgical residencies did not differ significantly in their scoring from students pursuing nonsurgical specialties.     

Outcomes of primary fascial closure after open abdomen for nontrauma emergency general surgery patients

Background

Emergency general surgery patients are increasingly being managed with an open abdomen (OA). Factors associated with complications after primary fascial closure (PFC) are unknown.

Methods

Demographic and operative variables for all emergency general surgery patients managed with OA at an academic medical center were prospectively examined from June to December 2013. Primary outcome was complication requiring reoperation.

Results

Of 58 patients, 37 managed with OA achieved PFC. Of these, 14 needed re-exploration for dehiscence, compartment syndrome, infection, or other. Complications after PFC were not associated with age, type of operative intervention, time to closure, re-explorations, comorbidities, or mortality. Complications correlated with higher body mass index (P = .02), skin closure (P = .04), plasma infusion (P = .01), and less intraoperative bleeding (P = .05). Deep surgical site infection correlated with fascial dehiscence (P = .02).

Conclusions

Reoperation after PFC was more likely in obese and nonhemorrhagic patients. Recognition of these factors and strategies to reduce surgical site infection may improve outcomes.     

Porcine wet lab improves surgical skills in third year medical students

Background

Medical students desire to become proficient in surgical techniques and believe their acquisition is important. However, the operating room is a challenging learning environment. Small group procedural workshops can improve confidence, participation, and performance. The use of fresh animal tissues has been rated highly among students and improves their surgical technique. Greater exposure to surgical procedures and staff could positively influence students' interest in surgical careers. We hypothesized that a porcine “wet lab” course for third year medical students would improve their surgical skills.

Methods

Two skills labs were conducted for third year medical students during surgery clerkships in the fall of 2011. The students' surgical skills were first evaluated in the operating room across nine dimensions. Next, the students performed the following procedures during the skills lab: (1) laparotomy; (2) small bowel resection; (3) splenectomy; (4) partial hepatectomy; (5) cholecystectomy; (6) interrupted abdominal wall closure; (7) running abdominal wall closure; and (8) skin closure. After the skills lab, the students were re-evaluated in the operating room across the same nine dimensions. Student feedback was also recorded. Fifty-one participants provided pre- and post-lab data for use in the final analysis.

Results

The mean scores for all nine surgical skills improved significantly after participation in the skills lab (P ≤ 0.002). Cumulative post-test scores also showed significant improvement (P = 0.002). Finally, the student feedback was largely positive.

Conclusions

The surgical skills of third year medical students improved significantly after participation in a porcine wet lab, and the students rated the experience as highly educational. Integration into the surgery clerkship curriculum would promote surgical skill proficiency and could elicit interest in surgical careers.     

Cutting too deep? Assessing the impact of a shorter surgery clerkship on students' clinical skills and knowledge

Background

The aim of this study was to compare the performance of students completing an 8-week versus a 6-week surgery clerkship on an objective structured clinical examination (OSCE) and the National Board of Medical Examiners (NBME) clinical science surgery examination.

Methods

One hundred fifteen students from the 8-week clerkship and 99 from the 6-week clerkship were included. Performance on a summative OSCE was assessed using behaviorally anchored checklists. NBME exams were graded using the NBME's standard scaled scores. Results were compared using 2-tailed, independent-samples, unequal-variance t tests.

Results

Mean OSCE scores for the 8-week and 6-week curricula were not statistically different. Mean NBME scores also did not statistically differ. Six-week students performed significantly better in the specific OSCE subdomains of blood pressure, orthostatic blood pressure, rectal exam, and fecal occult blood test.

Conclusions

Overall OSCE and NBME exam performance did not differ between 8-week and 6-week surgery clerkship students.     

A mentorship-based preclinical elective increases exposure,confidence, and interest in surgery

Background

The predicted shortage of surgeons is of growing concern with declining medical student interest in surgical careers. We hypothesized that earlier exposure to operative experiences and the establishment of resident mentors through a preclinical elective would enhance student confidence and interest in surgery.

Methods

We developed a preclinical elective in surgery, which served as an organized curriculum for junior medical students to experience surgery through a paired resident-mentorship model. We assessed student exposure and confidence with clinical activities before and after the elective (N = 24, 100% response rate). We compared these students with a cohort of peers not enrolled in the elective (N = 147, 67% response rate).

Results

We found significantly improved confidence (2.8 vs 4.4) and clinical exposure (2.4 vs 4.3) before versus after the elective, with precourse scores equal to matched peers.

Conclusions

This elective incorporates elements that have been shown to positively influence student decision making in surgical career choice. The mentorship model promotes residents as educators, whereas the elective provides a means for early identification of students interested in surgery.     

Outcomes after resection of cortisol-secreting adrenocortical carcinoma

Background

We sought to define the impact of cortisol-secreting status on outcomes after surgical resection of adrenocortical carcinoma (ACC).

Methods

The U.S ACC group database was queried to identify patients who underwent ACC resection between 1993 and 2014. The short-term and long-term outcomes were assessed.

Results

The incidence of all functional and cortisol-secreting tumors was 40.6% and 22.6%, respectively. On multivariable analysis, cortisol secretion remained associated with an increased risk of postoperative complications (odds ratio = 2.25, 95 % confidence interval = 1.04 to 4.88; P = .04). At a median follow-up of 17.6 months, 118 patients (50.4%) had developed a recurrence. On multivariable analysis, after adjusting for patient and disease-related factors cortisol secretion independently predicted shorter recurrence-free survival (Hazard ratio = 2.05, 95% confidence interval = 1.16 to 3.60; P = .01).

Conclusions

Cortisol secretion was associated with an increased risk of postoperative morbidity. Recurrence remains high among patients with ACC after surgery; cortisol secretion was independently associated with a shorter recurrence-free survival. Tailoring postoperative surveillance of ACC patients based on their cortisol secreting status may be important.