Awareness and usage of evidence-based learning strategies among health professions students and faculty

Abstract

Introduction: Learning is essential and life-long for faculty and students. Often students and teachers use ineffective learning strategies and are not aware of evidence-based strategies.

Methods: A multicenter, international, cross-sectional, online survey-based assessment of awareness of evidence-based learning strategies among health professions students (n?=?679) and faculty (n?=?205).

Results: Students endorsed many study habits which violate evidence-based principles, including studying whatever is due soonest (389/679, 57%), failing to return to course material once a course has ended (465/679, 68%), and re-reading underlined or highlighted notes (298.679, 44%). While the majority of faculty surveyed (125/157, 80%) reported recommending effective study strategies for their students, most students (558/679, 82%) said they did not study the way they do because of instruction from faculty. The majority of faculty (142/156, 91%) and students (347/661, 53%) believe students have different learning styles.

Discussion: The results of this study demonstrate health professions students continue to use many ineffective study strategies, and both students and faculty hold misconceptions about evidence-based learning. While planning a curriculum, medical educators should focus on teaching students how to learn and use higher order thinking procedures in addition to teaching content.     

Taking stock of what is known about faculty development in competency-based medical education: A scoping review paper

Abstract

Purpose: The primary objective was to inventory what is currently known about faculty development (FD) for competency-based medical educations (CBME) and identify gaps in the literature.

Methods: A scoping review methodology was employed. Inclusion criteria for article selection were established with two reviewers completing a full-text analysis. Quality checks were included, along with iterative consultation on data collection and consensus decision making via a grounded theory approach.

Results: The review identified 19 articles published between 2009 and 2018. Most articles (N?=?15) offered suggestions as to what should happen with FD in CBME, but few (N?=?4) adopted an experimental design. Six main themes were identified with three main features of FD noted across themes: (1) The importance of direct and timely feedback to faculty members on their teaching and assessment skills. (2) The role of establishing shared mental models for CBME curricula. (3) That FD is thought of longitudinally, not as a one-time bolus.

Conclusion: This work illustrates that there is limited, high quality research in FD for CBME. Future FD activities should consider employing a longitudinal and multi-modal program format that includes feedback for the faculty participants on their teaching and assessments skills, including the development of faculty coaching skills.     

Promoting resiliency for interprofessional faculty and senior medical students: Outcomes of a workshop using mind-body medicine and interactive reflective writing

Background: Health care professions faculty/practitioners/students are at risk for stress and burnout, impacting well-being, and optimal patient care.

Aims: We conducted a unique intervention: an interprofessional, experiential, skills-based workshop (IESW) combining two approaches: mind-body medicine skills and interactive reflective writing (RW) fostering self-awareness, self-discovery, reflection, and meaning-making, potentially preventing/attenuating burnout and promoting resiliency.

Methods: Medical and nursing faculty and senior medical students (N?=?16) participated in a 2-hour workshop and completed (1) Professional Quality of Life measure (ProQOL) and (2) a questionnaire evaluating understanding of professional burnout and resiliency and perceived being prepared to apply workshop techniques. Thematic analyses of anonymized RWs exploring meaningful clinical or teaching experiences were conducted.

Results: Participants reported better understanding of professional burnout/resiliency and felt better prepared to use meditation and RW as coping tools. RW themes identified experiencing/grappling with a spectrum of emotions (positive and negative) as well as challenge and triumph within clinical and teaching experiences as professionally meaningful.

Conclusions: Positive outcomes were obtained within a synergistic resiliency skills building exercise. Successful implementation of this IESW provides good rationale for studying impact of this intervention over a longer period of time, especially in populations with high rates of stress and burnout.     

Comparison of faculty assessment and students’ self-assessment of performance during clinical case discussions in a pharmacotherapy capstone course

Objectives: The primary objective of this study was to compare faculty assessment and third year students' self-assessment of performance in clinical case discussions. The secondary objective was to evaluate if student characteristics influence self-assessments.

Methods: This retrospective analysis compared faculty and student self-assessment scores for two clinical case discussions using Spearman’s correlation and Wilcoxon’s signed ranks test. Chi-squared test was used to compare frequency of faculty and student self-assessments indicating the highest possible rating for the pooled score and for each individual component. The pooled score included three individual components: level of engagement, quality of contribution, and professionalism.

Results: Pooled faculty and student self-assessments correlated for both the first (r?=?0.41, p?r?=?0.35; p?p?=?0.25) and second (58.6% vs. 47.4%, p?=?0.05) clinical case discussions. Student characteristics (age, gender, and grade point average at graduation) did not influence self-assessments.

Conclusions: Students’ self-assessment correlated with faculty assessment of performance during clinical case discussions. Increased use of self-assessments for professional development in pharmacy and other healthcare professional curricula should be considered.     

Evaluation of medical record quality and communication skills among pediatric interns after standardized parent training history-taking in China

Objective: The effect of using standardized parent training history-taking on the quality of medical records and communication skills among pediatric interns was determined.

Methods: Fifth-year interns who were undertaking a pediatric clinical practice rotation were randomized to intervention and control groups. All of the pediatric interns received history-taking training by lecture and bedside teaching. The pediatric interns in the intervention group also received standardized parent history-taking training. The following two outcome measures were used: the scores of medical records, which were written by the pediatric interns after history-taking from real parents of pediatric patients; and the communication assessment tool (CAT) assessed by real parents.

Results: The general information, history of present illness (HPI), past medical history, personal history, family history, diagnosis, diagnostic analysis, and differential diagnosis scores in the intervention group were significantly higher than the control group (p?Conclusions: Standardized parent training history-taking is effective in improving the quality of medical records by pediatric interns. Standardized parent training history-taking is a superior teaching tool for clinical reasoning ability, as well as communication skills in clinical pediatric practice.     

Small group activities within academic communities improve the connectedness of students and faculty

Background: The University of California, San Diego, School of Medicine implemented a curriculum change that included reduction of lectures, incorporation of problem-based learning and other small group activities. Six academic communities were introduced for teaching longitudinal curricular content and organizing extracurricular activities.

Methods: Surveys were collected from 904 first- and second-year medical students over 6 years. Student satisfaction data with their sense of connectedness and community support were collected before and after the implementation of the new curriculum. In a follow-up survey, medical students rated factors that contributed to their sense of connectedness with faculty and students (n?=?134).

Results: Students’ perception of connectedness to faculty significantly increased following implementation of a curriculum change that included academic communities. Students ranked small group clinical skills activities within academic communities significantly higher than other activities concerning their sense of connectedness with faculty. Students’ perception of connectedness among each other was high at baseline and did not significantly change. Small group activities scored higher than extracurricular activities regarding students’ connectedness among themselves.

Conclusions: The implementation of a new curriculum with more small group educational activities including academic communities enhanced connectedness between students and faculty and resulted in an increased sense of community.     

“But Dr Google said…” – Training medical students how to communicate with E-patients

Abstract

Purpose: Patients who have access to information online may feel empowered and also confront their physicians with more detailed questions. Medical students are not well-prepared for dealing with so-called “e-patients.” We created a teaching module to deal with this, and evaluate its effectiveness.

Method: Senior medical students had to manage encounters with standardized patients (SPE) in a cross-over design. They received blended-learning teaching on e-patients and a control intervention according to their randomization group (EI/LI?=?early/late intervention). Each SPE was rated by two blinded video raters, the SP and the student.

Results: N?=?46 students could be included. After the intervention, each group (EI, LI) significantly improved their competency in dealing with e-patients as judged by expert video raters (EI: M_T0?=?9.75 (2.51) versus M_T1?=?16.60 (2.80); LI: M_T0?=?8.70 (2.14) versus M_T2?=?15.20 (2.84); both p < 0.001) and SP (EI: M_T0?=?24.13 (4.83) versus M_T1?=?26.52 (3.06); LI: M_T0?=?23.37 (3.10) versus M_T2?=?27.47 (4.38); both p < 0.001). Students’ rating showed a similar non-significant trend.

Conclusions: Students, SP and expert video raters determined that blended-learning teaching can improve students’ competencies when dealing with e-patients. Within the study period, this effect was lasting; however, further studies should look at long-term outcomes.     

Teachers’ mindsets in medical education: A pilot survey of clinical supervisors

Abstract

Purpose: Current medical education models maintain that competencies such as professionalism and communication can be taught; however, some argue that certain attributes that make up these competencies, such as empathy, are fixed. Teachers’ implicit theories, or mindsets (beliefs about the fixed versus learnable nature of human attributes) have been shown to impact their teaching and assessment practices; but little work has explored mindsets in medical education. We examined clinical supervisors’ mindsets of two cognitive attributes (intelligence and clinical reasoning) and two affective attributes (moral character and empathy).

Methods: Clinical supervisors (n?=?40) from three specialities completed a survey designed to measure mindsets using two existing instruments for intelligence and moral character and 18 new items for clinical reasoning and empathy. Participants completed the survey twice for test-retest reliability (n?=?25).

Results: New items had satisfactory psychometric properties. Clinical supervisors’ mindsets were mixed. Only 8% of participants saw clinical reasoning as fixed while more saw empathy (45%), intelligence (53%), and moral character (53%) as fixed – running counter to current educational models that characterize these attributes as learnable.

Conclusion: This study provides evidence supporting the use of these new tools to measure mindsets that may help to better understand the impact of mindsets on medical education.     

Impact of sociodemographic characteristics of applicants in multiple mini-interviews

Background: Multiple mini-interviews (MMI) are commonly used for medical school admission. This study aimed to assess if sociodemographic characteristics are associated with MMI performance, and how they may act as barriers or enablers to communication in MMI.

Methods: This mixed-method study combined data from a sociodemographic questionnaire, MMI scores, semi-structured interviews and focus groups with applicants and assessors. Quantitative and qualitative data were analyzed using multiple linear regression and a thematic framework analysis.

Results: 1099 applicants responded to the questionnaire. A regression model (R^2?=^?0.086) demonstrated that being age 25–29 (β?=?0.11, p?=?0.001), female and a French-speaker (β?=?0.22, p?=?0.003) were associated with better MMI scores. Having an Asian-born parent was associated with a lower score (β?=??0.12, p?Conclusion: Age, gender, ethnicity, socioeconomic status and language seem to be associated with applicants’ MMI scores because of perceived differences in communications skills and life experiences. Monitoring this association may provide guidance to improve fairness of MMI stations.     

Assessing learning in the adaptive curriculum

Abstract

Background: Medical education is a dynamic process that will continuously evolve to respond to changes in the foundations of medicine, the clinical practice of medicine and in health systems science.

Purpose: In this paper, we review how assessing learning in such a dynamic environment requires comprehensive flexible and adaptable methodological approaches designed to assess knowledge attainment and transfer, clinical skills/competency development, and ethical/professional behavior. Adaptive assessments should measure the learner’s ability to observe where changes in health care delivery are needed and how to implement them. Balancing formative and summative assessments will promote reflective learning so that each student will reach her/his highest potential. From the programmatic perspective, measuring the design and delivery of instruction in relation to students? efforts to achieve competency will improve learning and foster continuous professional development of faculty and advance the science of learning.

Approach: We describe how two medical schools are approaching adaptive assessment, including using portfolio systems that encompass teaching and learning experiences while offering real-time longitudinal tracking of digital data toward improving learning and provide curricula continuous improvement cycles. Using latest technologies, portfolios produce actionable data displays with precise guidance for learning and program development.     

A multi-source feedback tool for measuring a subset of Pediatrics Milestones

Background: The Pediatrics Milestones Assessment Pilot employed a new multisource feedback (MSF) instrument to assess nine Pediatrics Milestones among interns and subinterns in the inpatient context.

Objective: To report validity evidence for the MSF tool for informing milestone classification decisions.

Methods: We obtained MSF instruments by different raters per learner per rotation. We present evidence for validity based on the unified validity framework.

Results: One hundred and ninety two interns and 41 subinterns at 18 Pediatrics residency programs received a total of 1084 MSF forms from faculty (40%), senior residents (34%), nurses (22%), and other staff (4%). Variance in ratings was associated primarily with rater (32%) and learner (22%). The milestone factor structure fit data better than simpler structures. In domains except professionalism, ratings by nurses were significantly lower than those by faculty and ratings by other staff were significantly higher. Ratings were higher when the rater observed the learner for longer periods and had a positive global opinion of the learner. Ratings of interns and subinterns did not differ, except for ratings by senior residents. MSF-based scales correlated with summative milestone scores.

Conclusion: We obtain moderately reliable MSF ratings of interns and subinterns in the inpatient context to inform some milestone assignments.     

Clinical teaching performance improvement of faculty in residency training: A prospective cohort study

Purpose: The purpose of this study is to investigate how aspects of a teaching performance evaluation system may affect faculty’s teaching performance improvement as perceived by residents over time.

Methods: Prospective multicenter cohort study conducted in The Netherlands between 1 September 2008 and 1 February 2013. Nine hundred and one residents and 1068 faculty of 65 teaching programs in 16 hospitals were invited to annually (self-) evaluate teaching performance using the validated, specialty-specific System for Evaluation of Teaching Qualities (SETQ). We used multivariable adjusted generalized estimating equations to analyze the effects of (i) residents’ numerical feedback, (ii) narrative feedback, and (iii) faculty’s participation in self-evaluation on residents’ perception of faculty’s teaching performance improvement.

Results: The average response rate over three years was 69% for faculty and 81% for residents. Higher numerical feedback scores were associated with residents rating faculty as having improved their teaching performance one year following the first measurement (regression coefficient, b: 0.077; 95% CI: 0.002–0.151; p?=?0.045), but not after the second wave of receiving feedback and evaluating improvement. Receiving more suggestions for improvement was associated with improved teaching performance in subsequent years.

Conclusions: Evaluation systems on clinical teaching performance appear helpful in enhancing teaching performance in residency training programs. High performing teachers also appear to improve in the perception of the residents.     

Which professional (non-technical) competencies are most important to the success of graduate veterinarians? A Best Evidence Medical Education (BEME) systematic review: BEME Guide No. 38

Background: Despite the growing prominence of professional (non-technical) competencies in veterinary education, the evidence to support their importance to veterinary graduates is unclear.

Aim: To summarize current evidence within the veterinary literature for the importance of professional competencies to graduate success.

Methods: A systematic search of electronic databases was conducted (CAB Abstracts, Web of Science, PubMed, PsycINFO, ERIC, Australian and British Education Index, Dissertations & Theses) from 1988 to 2015 and limited to the veterinary discipline (veterinar* term required). Evidence was sought from consensus-based competence frameworks, surveys of stakeholder perceptions, and empirical evidence linked to relevant outcomes (e.g. employability, client satisfaction or compliance). Data extraction was completed by two independent reviewers and included a quality assessment of each source.

Results: Fifty-two sources were included in the review, providing evidence from expert frameworks (10 sources), stakeholder perceptions (30 sources, including one from the previous category), and empirical research (13 sources). Communication skills were the only competency to be well-supported by all three categories of evidence. Other competencies supported by multiple sources of empirical evidence include empathy, relationship-centered care, self-efficacy, and business skills. Other competencies perceived to be relatively more important included awareness of limitations, professional values, critical thinking, collaboration, and resilience.

Conclusions: This review has highlighted the comparatively weak body of evidence supporting the importance of professional competencies for veterinary graduate success, with the exception of communication skills. However we stress this is more indicative of the scarcity of high-quality veterinary-based education research in the field, than of the true priority of these competencies.     

A unique flipped classroom approach shows promising results in physician assistant education

Abstract

Aims: The flipped classroom (FC) model utilizes pre-class foundational learning with in-class higher-order application exercises. FC approaches have reported positive perceptions and mixed academic performance outcomes in medical education; however, little evidence exists on their impact in the 56-week didactic curriculum of Physician Assistant (PA) education.

Methods: A 4-week FC curriculum was piloted in a PA behavioral medicine course (n?=?34). Before class, students completed online video modules. During class, students completed a pre-class quiz and answered clinical case-based questions in a think-pair-share format.

Results: Students rated in-class time higher than a lecture-based segment of the same course (p?=?0.028). Assessment of achievement emotions demonstrated increased enjoyment (p?=?0.028) and decreased boredom (p?<?0.001) in the FC curriculum with no difference in anxiety. The FC curriculum produced a 57.2?min increase in pre-class preparation time with no change in post-class studying time. Compared to historical lecture-based controls (n?=?35), students in the FC curriculum scored significantly higher on clinical vignette questions (p?=?0.019) with equivalent performance on other question formats.

Conclusions: This study offers the first positive outcomes of an FC approach in PA education and provides a framework for improving academic performance while scaling back in-class time.     

Defining curricular priorities for student-as-teacher programs: A National Delphi Study

Background: “Student-as-Teacher” (SaT) programs have been growing in number to prepare medical students for their teaching roles in residency and beyond, but it remains unknown what content areas should be covered in SaT curricula.

Aim: To determine five to ten “essential” content areas for inclusion in SaT curricula using expert opinion.

Methods: Using a three-round Delphi process, moderators iteratively surveyed a panel of 28 medical educators (25 academy directors and three individuals identified as having expertise in undergraduate medical education) representing 25 medical schools in the United States. This “SaT Delphi Working Group” was tasked with rating topics for inclusion in SaT curricula on a 3-point scale (i.e. 1. “essential,” 2. “important, but not essential” 3. “not important”). Topics achieving ≥70% consensus as “essential,” “important” or “not important” were accepted by the moderators and removed from subsequent rounds.

Results: Hundred per cent response rate (n?=?28) was achieved for all survey rounds. Five content areas reached consensus as “essential” for inclusion in a SaT curriculum: feedback, bedside teaching and clinical precepting, small-group teaching, case-based teaching and professionalism as a medical educator.

Conclusion: This consensus from a group of leaders in medical education is a first step toward the implementation of more developmentally-appropriate SaT competencies.     

Faculty development: From rubies to oak

Abstract

Introduction: Faculty development has played a significant role in health professions education over the last 40?years. The goal of this perspective is to present a portrait of faculty development in Medical Teacher since its inception and to highlight emerging trends moving forward.

Method: All issues of Medical Teacher were reviewed, using the search terms faculty development, staff development, professional development, or in-service training for faculty. The search yielded 286 results of which 145 focused specifically on faculty development initiatives, reviews, or frameworks.

Findings: This review demonstrated a significant growth in publications related to faculty development in Medical Teacher over the last 40?years, with a primary focus on teaching improvement and traditional approaches to faculty development, including workshops, short courses and other structured, group activities. The international nature of faculty development was also highlighted.

Recommendations: Moving forward, it is suggested that we: broaden the scope of faculty development from teaching to academic development; expand our approaches to faculty development, to include peer coaching, workplace learning and communities of practice; utilize a competency-based framework to guide the development of faculty development curricula; support teachers’ professional identities through faculty development; focus on organizational development and change; and rigorously promote research and scholarship in faculty development.     

Use of an objective structured clinical exam (OSCE) for early identification of communication skills deficits in interns

Introduction: There is limited information about whether OSCE during GME orientation can identify trainee communication deficits before these become evident via clinical performance evaluations.

Methods: Ninety-seven interns matriculating to eight residency programs in six specialties at four hospitals participated in a nine-station communication skills OSCE. Ratings were based on the “Kalamazoo, adapted” communication skills checklist. Possible association with intern performance evaluations was assessed by repeated-measures logistic regression and ROC curves were generated.

Results: The mean OSCE score was 4.08?±?0.27 with a range of 3.3–4.6. Baseline OSCE scores were associated with subsequent communication concerns recorded by faculty, based on 1591 evaluations. A 0.1-unit decrease in the OSCE communication score was associated with an 18% higher odds of being identified with a communication concern by faculty evaluation (odds ratio 1.18, 95% CI 1.01–1.36, p?=?0.034). ROC curves did not demonstrate a “cut-off” score (AUC=?0.558). Non-faculty evaluators were 3–5 times more likely than faculty evaluators to identify communication deficits, based on 1900 evaluations.

Conclusions: Lower OSCE performance was associated with faculty communication concerns on performance evaluations; however, a “cut-off” score was not demonstrated that could identify trainees for potential early intervention. Multi-source evaluation also identified trainees with communication skills deficits.     

Twelve tips for successfully implementing logbooks in clinical training

Background: Logbooks are widely used to set learning outcomes and to structure and standardize teaching in clinical settings. Experience shows that logbooks are not always optimally employed in clinical training. In this article, we have summarized our own experiences as well as results of studies into twelve tips on how to successfully implement logbooks into clinical settings.

Methods: We conducted both a workshop concerning the importance of logbook training to exchange experiences in teaching practice, organization, didactic knowledge and a literature research to compare our own experiences and add additional aspects.

Results: Tips include the process of developing the logbook itself, the change-management process, conditions of training and the integration of logbooks into the curriculum.

Conclusions: Logbooks can be a valuable tool for training in clinical settings, especially when multiple sites are involved, when you take our tips into consideration.     

Defining leadership competencies for pediatric critical care fellows: Results of a national needs assessment

Introduction: Physicians in training, including those in Pediatric Critical Care Medicine, must develop clinical leadership skills in preparation to lead multidisciplinary teams during their careers. This study seeks to identify multidisciplinary perceptions of leadership skills important for Pediatric Critical Care Medicine fellows to attain prior to fellowship completion.

Methods: We performed a multi-institutional survey of Pediatric Critical Care Medicine attendings, fellows, and nurses. Subjects were asked to rate importance of 59 leadership skills, behaviors, and attitudes for Pediatric Critical Care practitioners and to identify whether these skills should be achieved before completing fellowship. Skills with the highest ratings by respondents were deemed essential.

Results: Five hundred and eighteen subjects completed the survey. Of 59 items, only one item (“displays honesty and integrity”) was considered essential by all respondents. When analyzed by discipline, nurses identified 21 behaviors essential, fellows 3, and attendings 1 (p?p?Conclusions: Despite significant variability among Pediatric Critical Care attendings, fellows, and nurses in identifying which clinical leadership competencies are important for graduating Pediatric Critical Care fellows, they place the highest importance on skills in self-management and self-awareness. Leadership skills identified as most important may guide the development of interventions to improve trainee education and interprofessional care.