Differences in operative self-assessment between male and female plastic surgery residents: A survey of 8,149 cases

BackgroundPrevious studies show female residents tend to underrate and male residents to overrate their own performance. We sought to determine if plastic surgery resident trainee self-evaluations differ by resident sex.MethodsWe extracted Operative Entrustability Assessment (OEA) data for plastic surgery programs from MileMarker?, a program capable of storing assessment data for CPT-coded procedures. Complete OEAs contain a trainee self-assessment and attending surgeon assessment. We used simple statistics and linear regression to assess differences, stratifying by trainee sex and post-graduate year (PGY).ResultsWe analyzed 8149 OEAs from 3 training programs representing 64 residents (25% female) and 51 attendings. Compared to attending assessments, both male and female residents significantly underrated their performance during PGY1. However, during PGY2-6 male residents’ self-evaluations were significantly higher and female residents’ self-evaluations significantly lower than their attending evaluations.ConclusionsResults demonstrated female plastic surgery residents underestimated and male residents overestimated their performance. Further studies are needed to determine reasons for these differences.     

Anesthesiology resident personality type correlates with faculty assessment of resident performance

Study ObjectiveTo study the association between anesthesiology residents' personality preference types, faculty evaluations of residents' performance, and knowledge.DesignConvenience sample and prospective study.SettingAcademic department of anesthesiology.SubjectsConsenting anesthesiology residents (n = 36).InterventionsAll participants completed the Myers Briggs Type Indicator® (MBTI®).MeasurementsAll residents' 6-month summation of daily focal evaluations completed by faculty [daily performance score (DPS); 1 = unsatisfactory, 2 = needs improvement, 3 = meets expectations, 4 = exceeds expectations], as well as a global assessment of performance (GAP) score based on placement of each resident into perceived quartile compared with their peers (ie,1 = first, or top, quartile) by senior faculty (n = 7) who also completed the MBTI, were obtained. The resident MBTI personality preferences were compared with the DPS and GAP scores, the United States Medical Licensing Examination (USMLE) I and II scores, and faculty MBTI personality type.Main ResultsThere was no association between personality preference type and performance on standardized examinations (USMLE I, II). The mean GAP score was better (higher quartile score) for Extraverts than Introverts (median 2.0 vs 2.6, P = 0.0047) and for Sensing versus Intuition (median 2.0 vs 2.6, P = 0.0206) preference. Faculty evaluator MBTI preference type did not influence the GAP scores they assigned residents. Like GAP, the DPS was better for residents with Sensing versus Intuition preference (median 3.5 vs 3.3, P = 0.0111). No difference in DPS was noted between Extraverts and Introverts.ConclusionsPersonality preference type was not associated with resident performance on standardized examinations, but it was associated with faculty evaluations of resident performance. Residents with Sensing personality preference were evaluated more favorably on global and focal faculty evaluations than those residents who chose the Intuition preference. Extraverted residents were evaluated more favorably on global but not focal assessment of performance.     

Do Men Outperform Women During Orthopaedic Residency Training?

Background

Orthopaedic surgery residency has one of the lowest percentages of women (13.1%) of all primary surgical specialties. There are many possible reasons for this, including bias during the selection process.

Questions/purposes

We therefore asked whether performance during residency might adversely bias the selection of future female orthopaedic residents by researching whether males and females perform equally in orthopaedic surgery residency.

Methods

Ninety-seven residents enrolled in our residency between 1999 and 2009; six males and one female left the program, leaving 90 residents (73 males, 17 females) as the study cohort. Resident performance was compared for OITE scores, ABOS results, faculty evaluations, and in a resident graduate survey.

Results

Males and females had similar faculty evaluations in all ACGME competency areas. Males and females had similar mean OITE scores for Years 2–5 of residency, although males had higher mean scores at Years 3 through 5. Males and females had similar mean ABOS Part 1 scores and ABOS Part 1 pass rates; however, fewer males than females took more than one attempt to pass. Males and females had similar Part 2 pass rates or attempts. For the 45 resident graduates surveyed, females pursued fellowships equally to males, worked slightly less hours in practice, and reported higher satisfaction with their career choice.

Conclusions

For the 90 residents at one residency program, we observed no differences between males’ and females’ performance. Although females pursue orthopaedic residency less frequently than males, performance during residency should not bias their future selection.     

Is there a gender bias in milestones evaluations in general surgery residency training?

BackgroundStudies of gender disparity in surgical training have yielded conflicting results. We hypothesize that there is no influence of gender on resident self-evaluation Milestone (SEM) scores and those assigned by the Clinical Competency Committee (CCC).Methods42 residents (25 male & 17 female) and faculty completed 300 Accreditation Council for Graduate Medical Education (ACGME) Milestone evaluations over a 4-year period. Two-way ANOVA, intraclass correlations coefficients, and general linear mixed models were used for analysis.ResultsCCC Milestone scores from 150 evaluations, 51 (34%) for female residents and 99 (66%) for male residents, were compared to corresponding SEM scores. There is a high interrater reliability (self vs. CCC). There was a significant increase in scores with advancing PGY levels (p < 0.001). No effect of gender on Milestones scores (p > 0.05) was noted.ConclusionsWe found no significant differences in Milestones scores between male and female residents as determined by the CCC. Both scores improved significantly as residents progressed in training.     

Is there gender bias in the evaluation of surgical residents?

Although it might be assumed that the performance of surgical residents is assessed by faculty without regard to gender, no study to date has been undertaken to evaluate this. for 19 years, the Department of Surgery in our institution has had female faculty members to evaluate residents and, with the exception of 1969, female residents to be evaluated. All residents are evaluated by supervisory faculty after each clinical rotation. The same evaluation process has been used throughout this time, assessing residents on a five-point scale (1 = best) in each of six attributes: ethics, judgment, technical skills, knowledge, interpersonal skills, and work habits. The faculty evaluations of general surgery residents from academic years 1967 to 1985 were reviewed. A total of 2356 evaluations of 144 residents (22% female) by 69 faculty members (22% female) was available. For valid comparison, a subset of 702 evaluations was selected. Included were only those evaluations in which residents at the same level of training during the same academic year were evaluated by the same faculty members and in which both the resident and faculty groups had both male and female members. The average scores for male and female residents were nearly identical as determined by male and female faculty members, and no differences were statistically significant. In our program, with more than 20% women residents and faculty members over a 19-year period, no bias related to gender was demonstrated.     

To see ourselves as others see us: self-assessment in surgical residency

PurposeAlthough self-assessment is a widely used educational technique, the value of self-evaluation in surgical residency has not been clearly defined. This study was undertaken to assess the ability of residents to evaluate themselves using the same standards as the surgical faculty and to determine how this information may be used in a surgical training program.MethodsCategorical surgical residents were asked to grade themselves (scale, 1–10) in 12 performance characteristics by completing the same forms used by the faculty for 3 quarterly evaluation periods. Mean faculty grades ± standard deviation were computed for each resident and compared with resident self-evaluation grades for a global rating scale and for specific performance characteristics. Comparisons were made by Pearson correlation analysis, analysis of variance, and Bonferroni’s multiple comparisons test with significance accepted at p < 0.05.ResultsA significant correlation (r = 0.47; p < 0.0098) was identified between faculty and resident in the global score and in the specific performance characteristics of knowledge (r = 0.51; p = 0.0052), clinical judgement (r = 0.48; p = 0082), and technical ability (r = 0.52; p = 0.0036). For residents who accurately estimated their scores, mean faculty score was 7.3 ± 0.64, which was significantly higher than the mean score in overestimators (6.4 ± 0.89; p < 0.05), which in turn was significantly lower than the mean score in underestimators (8.2 ± 0.58; p < 0.001). No significant correlation was obtained for ethical standards and interpersonal relationships or between PGY level and accuracy of self-evaluation.ConclusionsWhereas resident self-evaluations correlated positively with faculty ratings both globally and in certain performance categories, the majority of residents over- or underestimated their abilities. Information from such self-evaluations may be useful in counseling residents as well as in monitoring and improving the evaluation process.     

Emotional intelligence and the relationship to resident performance: a multi-institutional study

Study ObjectiveTo test the hypothesis that emotional intelligence, as measured by a BarOn Emotional Quotient Inventory (EQ-i), the 125-item version personal inventory (EQ-i:125), correlates with resident performance.DesignSurvey (personal inventory) instrument.SettingFive U.S. academic anesthesiology residency programs.ParticipantsPostgraduate year (PGY) 2, 3, and 4 residents enrolled in university-based anesthesiology residency programs.MeasurementsResidents confidentially completed the BarOn EQ-i:125 personal inventory. The deidentified resident evaluations were sent to the principal investigator of a separate data collection study for data analysis. Data collected from the inventory were correlated with daily evaluations of the residents by residency program faculty. Results of the individual BarOn EQ-i:125 and daily faculty evaluations of the residents were compiled and analyzed.Main ResultsUnivariate correlation analysis and multivariate canonical analysis showed that some aspects of the BarOn EQ-i:125 were significantly correlated with, and likely to be predictors of, resident performance.ConclusionsEmotional intelligence, as measured by the BarOn EQ-i personal inventory, has considerable promise as an independent indicator of performance as an anesthesiology resident.     

Patient inequities in affording surgical resident operative autonomy at Veterans Affairs teaching hospitals,does it extend to female patients?

BackgroundExamining surgical resident operative autonomy within the Veterans Affairs (VA) System, we previously showed residents were afforded autonomy more frequently on Black patients. We hypothesized that, compared to males, female surgical patients receive less attending involvement and more resident autonomy during surgery.MethodsRetrospective review of all general/vascular surgeries performed at teaching VA hospitals from 2004 to 2019. Operative procedures are coded at the time of surgery as attending primary surgeon (AP), attending with resident (AR), or resident primary surgeon--attending not scrubbed (RP). The primary outcome was the difference in supervision rates between patient sexes.Results618,578 operations were examined—24.9% AP, 68.9% AR, and 6.2% RP. Overall, 5.9% of cases were performed on women. The rate of RP cases was higher in males compared to females (6.3% vs 5.3%, p < 0.001).ConclusionFemale veterans are less likely to have residents operate on them autonomously. Reasons for this require further characterization.     

Greater faculty familiarity with residents improves intraoperative entrustment

BackgroundLongitudinal contact between faculty and residents facilitates greater faculty entrustment. The purpose of this study is to assess the relationship between faculty familiarity with residents and faculty entrustment.Materials and methodsResearchers observed and rated entrustment behaviors using OpTrust, September 2015–June 2017 at Michigan Medicine. Faculty familiarity with resident was measured on a 1–4 scale (1 = not familiar, 4 = extremely familiar). ANOVA and Sidak adjusted multiple comparisons were used to assess the relationship between faculty familiarity and faculty entrustment.Results56 faculty and 73 residents were observed across 225 surgical cases. Faculty entrustment scores increased to 2.48 when resident familiarity was reported as “slightly familiar”. Faculty entrustment scores for “moderately familiar” increased to 2.57. Faculty entrustment scores for “extremely familiar” increased to 2.84.ConclusionsWe found a positive relationship between faculty familiarity and entrustment. These findings support greater continuity in faculty/resident relationships. Longitudinal contact allows learners to be granted progressive entrustment.SummaryThis study demonstrates a positive relationship between faculty familiarity with residents and an increase in intraoperative entrustment. These findings support greater continuity in faculty/resident relationships.     

Teaching across the continuum: variations in rankings and valued teaching components between surgery residents and medical students

Background

Surgical faculty teach medical students and residents within the same environment; however, each group may require different teaching methods. The aim of this study was to identify teaching components valued by these sets of learners.

Methods

Teaching evaluations for 43 surgical faculty members who received yearly evaluations from students and residents were analyzed. Highest and lowest ranked educators in medical student evaluations were traced to corresponding rankings in resident evaluations. Thematic analysis was conducted on written comments.

Results

Educators rated the highest in the medical student group were spread out among the residents' rankings, with several educators being rated the lowest. Similar patterns were observed in reverse for faculty rated highly by residents.

Conclusions

Residents and medical students value faculty teaching using different criteria, with residents more focused on operative autonomy and medical students focused on approachability. Using one group to define best teaching methods is insufficient as learners value different optimal attributes.     

Gender differences in the self-assessment of surgical residents

BACKGROUND: Despite equivalent performance as assessed by objective external measures, women tend to underestimate their abilities compared with their male counterparts. METHODS: The difference in ordinal values from faculty and resident self-evaluations was calculated for each general and plastic surgery resident. Objective external performance measures were compared for female and male residents. RESULTS: Male and female residents performed equivalently. All residents underestimated their abilities compared with faculty assessment; however, general surgery residents did so to a greater degree (P < .05). Female residents demonstrated a greater degree of underestimation than did their male colleagues; however, this was not statistically significant. CONCLUSIONS: Although female resident surgeons are generally confident in their abilities, this may be in contrast to the self-perception of many female medical students. Consideration of gender differences in self-perception may be important when providing feedback to female students and residents.     

Osteopathic medical students' perception of teaching effectiveness of their primary care clinical preceptors

BackgroundPrimary care clinical preceptors, including physicians practicing family medicine, general internal medicine and pediatrics, are important members of the osteopathic medical school faculty. They train, teach, and mentor medical students. Their role is crucial in achieving the mission of several osteopathic medical schools – that of preparing primary care physicians to provide quality patient care.ObjectiveWe investigated the perception of effective preceptor teaching by the evaluations of third and fourth year students from one osteopathic medical school. We sought to answer the following questions: 1) Do osteopathic medical students' perception of primary care clinical preceptors' teaching effectiveness differ among family medicine, internal medicine, and pediatrics sub-disciplines?; 2) Is there a difference in osteopathic medical students' perception of teaching effectiveness and perception of the amount of time they spend with their clinical preceptors?MethodsThe study participants included third and fourth year medical students that completed the E*Value Student Evaluation of Preceptor Form of primary care clinical teachers from August 2010 to June 2012 at an osteopathic medical school at a Midwestern university in the United States of America (USA). We performed analysis of variance (ANOVA) to gauge differences among the multiple categorical variables. Differences were considered statistically significant at p ≤ 0.05.ResultsA total of 1303 students completed the evaluations. We found that students' evaluations of all assigned preceptor-related criteria and all unassigned preceptor-related criteria were significant across the three sub-disciplines. This suggests that osteopathic medical students perceive the teaching of their primary care clinical preceptors as highly effective. We also found statistically significant differences among the three categories of time spent and the overall perception of teaching effectiveness of each of the preceptor categories.ConclusionEven though preceptor teaching effectiveness is only one aspect of clinical education, it is important to understand medical students' perception regarding strengths and weaknesses of their clinical training. Perception of preceptor teaching effectiveness by medical students can provide key insights to guide curriculum/instructional design and faculty development programming.     

Identification of promotion and prevention associated surgeon behaviors in the operating room to facilitate resident self-regulated learning

BackgroundThe regulatory focus theory (RFT) posits that people can pursue goals with a promotion or prevention focus. Greater alignment of RFT motivational styles between faculty and residents may enhance resident operative autonomy. This study establishes a set of faculty behaviors residents can identify to infer faculty motivational styles.Methods10 behaviors associated with promotion and prevention motivational styles were identified. General surgery residents rated faculty on how strongly they exhibit these behaviors. Faculty conducted a self-assessment of how strongly they exhibit these behaviors.ResultsThere is a positive correlation between resident and faculty ratings for the promotion-associated behaviors of “works quickly,” “high energy,” and “mostly provides broad oversight,” and for the prevention-associated behaviors of “works slowly and deliberately,” “quiet and calm,” and “preference for vigilant strategies.”ConclusionResidents can observe faculty operative behaviors to infer faculty motivational styles. Residents may use this knowledge to adjust to faculty motivational styles and enhance operative interactions.     

Letters of recommendation for surgical fellowships – Does gender make a difference?

BackgroundFemales comprise 1/3 of general surgery residents, 1/4 of surgical faculty and 10% of full professors. Inadequate sponsorship is one proposed mechanism for this decline. This study evaluated letters of recommendation (LOR) among applicants applying to a complex general surgical oncology (CGSO) fellowship.MethodsLinguistic analysis of LOR for CGSO applicants was conducted. Demographics of authors and features of the LOR were extracted. Differences by gender of the applicant were analyzed.ResultsAmong 340 letters, 67% were written for male and 33% written for female applicants. Males authored 84% of letters reviewed. Female authors used more grindstone adjectives than males (3.61 v 2.90). However, this difference was seen only among letters written for male applicants (3.82 v. 2.73). All other linguistic features were similar, aside from mention of physical appearance which was significantly more common in letters written about female applicants (4% v. 1%).ConclusionsFemale authors write substantively different letters than males. Physical appearance is a small but important difference in letters for female applicants.     

Educational feedback in the operating room: a gap between resident and faculty perceptions

BackgroundImmediate feedback regarding performance in the operating room remains a key component of resident education. The aim of this study was to assess resident and faculty perceptions regarding postoperative feedback.MethodsAnonymous surveys were distributed to residents and faculty members. Questions addressed the timing, amount, and specificity of feedback; satisfaction; and the definition and importance of feedback. Additional questions regarded the importance and frequency of feedback in 7 specific areas of surgical competency.ResultsResident satisfaction with timing, amount, and specificity of feedback was significantly lower than faculty satisfaction. Perceptions of the importance of feedback for each of the 7 specific areas did not differ. Faculty members' perceptions on the frequency of feedback were higher than residents' perception in all competencies of feedback (5-point scale, all P values = .001).ConclusionsThere are significant differences between resident and faculty perceptions regarding postoperative feedback. Although faculty members believed they delivered appropriate amounts of timely, quality feedback, this perception was not shared by residents.     

Development and emergency implementation of an online surgical education curriculum for a General Surgery program during a global pandemic: The University of Southern California experience

BackgroundPhysical distancing required by coronavirus disease 2019 (COVID-19) has limited traditional in-person resident education. We present our novel online curriculum for incorporation into traditional surgical educational programs.MethodsThe online curriculum utilized weekly sub-specialty themed faculty and resident created lectures, ABSITE practice questions, and weekly sub-specialty synchronized readings. Attendance, resident and faculty surveys, and completed ABSITE practice questions evaluated for curriculum success. Curriculum was adapted as COVID-19 clinical restructuring ended.Results77% and 80% of clinical residents attended faculty lectures and resident led topic discussions as compared to 66% and 48% attending traditional in-person grand rounds and SCORE curriculum (both p > 0.05). 71.9% of residents and 16.6% of faculty reported improved resident participation while none reported decreased levels of participation (p < 0.001). 87.1% of residents and 66.7% of faculty preferred the online curriculum (p = 0.374). Completed ABSITE practice questions per resident increased from 21 to 31 questions/week (p = 0.541).ConclusionOur online educational curriculum demonstrates success and can serve as a model for online restructuring of resident education.     

A comparison of anesthesiology resident and faculty electronic evaluations before and after implementation of automated electronic reminders

STUDY OBJECTIVE: To develop an automated e-mail reminder system to contact residents and faculty regarding incomplete evaluations. DESIGN, SETTING, INTERVENTION, AND MEASUREMENTS: In the retrospective study, two 9-month periods were evaluated representing pre- and post-introduction of the automated e-mail reminder system. Data collected contained the number of evaluations completed and the rating of residents and faculty at the University of Michigan Health System in 5 different categories on a 5-point scale. MAIN RESULTS: The use of electronic reminders resulted in a nearly 4-fold increase in the number of resident evaluations by faculty from 1050 to 3761. Faculty completing evaluations increased from 40 (61%) to 66 (100%). The mean evaluation scores showed statistically significant but clinically negligible change for resident judgement, interpersonal skills, and intraoperative management. Resident preoperative evaluation and knowledge did not show statistically significant changes. Residents completing evaluations of faculty increased from 244 to 1798, and the number of residents completing evaluations rose from 40 (56%) to 87 (100%). There were no statistically significant changes in the evaluation of faculty except in the category of feedback. CONCLUSION: An automated e-mail reminder system implemented to contact residents and faculty regarding incomplete evaluations for residents and faculty dramatically enhanced participation in the evaluation process.     

Surgeons underestimate their influence on medical students entering surgery

BackgroundPositive surgical role models influence medical students to pursue a career in surgery. However, the perception by role models of their own effectiveness has yet to be examined. In this study, we evaluated the influence of surgical role models on medical student career choice, and how these role models perceive themselves.MethodsWe distributed a voluntary and anonymous survey to third-year medical students, general surgery resident applicants, general surgery residents, and attending surgery faculty during the 2010–2011 academic year. We performed statistical analysis using the chi-square and Cochran–Mantel–Haenszel tests (P < 0.05 was significant).ResultsMedical students and resident applicants agreed that faculty and residents are important in shaping the career paths of students. The applicants were more likely to agree than were students that clerkship role models (P = 0.0049) and mentors (P = 0.0035) affected their interest in surgery. The applicants were also more likely to agree than the students that attending surgeons (P = 0.0004), senior (P = 0.0019) and junior (P = 0.0028) surgery residents served as positive role models. Although the surgical faculty and residents agreed with the students that each level of the surgical team served as positive role models, they did not agree as strongly with the students that they have an important role in shaping students' career path (P < 0.0001).ConclusionsSurgical faculty and residents serve as positive role models for medical students. They have an essential role in shaping students' career paths and should be more cognizant of their influence, which may draw a student toward or lead them away from the field of surgery.     

High vs. low entrustment behaviors in the operating room

BackgroundOperative experience with an appropriate degree of supervised autonomy is critical to resident training. Progressively greater intraoperative entrustment has been associated with gradually higher levels of resident autonomy. This study attempts to identify consistently observed intraoperative behaviors that are linked with higher resident entrustment.MethodsThis qualitative study analyzed observational notes recorded by trained raters who provided entrustment scores for 204 surgical cases at Michigan Medicine from 2015 to 2017. Notes were coded in NVivo12. Thematic analysis was used to identify themes and patterns within the data.ResultsThe analysis generated 144 codes. Codes were clustered into 10 themes. These themes manifested differently in intraoperative behaviors strongly associated with high entrustment versus low entrustment.ConclusionThis study demonstrates key differences in intraoperative behaviors exhibited by residents and faculty in high and low entrustment interactions. Awareness of behaviors that enhance entrustment can help faculty augment resident learning and enable higher resident operative autonomy.     

Multi-Disciplinary Trauma Evaluation and Management Simulation (MD-TEAMS) training for emergency medicine and general surgery residents

BackgroundSuccessful trauma resuscitation relies on multi-disciplinary collaboration. In most academic programs, general surgery (GS) and emergency medicine (EM) residents rarely train together before functioning as a team.MethodsIn our Multi-Disciplinary Trauma Evaluation and Management Simulation (MD-TEAMS), EM and GS residents completed manikin-based trauma scenarios and were evaluated on resuscitation and communication skills. Residents were surveyed on confidence surrounding training objectives.ResultsResidents showed improved confidence running trauma scenarios in multi-disciplinary teams. Residents received lower communication scores from same-discipline vs cross-discipline faculty. EM residents scored higher in evaluation and planning domains; GS residents scored higher in action processes; groups scored equally in team management. Strong correlation existed between team leader communication and resuscitative skill completion.ConclusionMD-TEAMS demonstrated correlation between communication and resuscitation checklist item completion and communication differences by resident specialty. In the future, we plan to evaluate training-related resident behavior changes and specialty-specific communication differences by residents.