Peer assessment in the objective structured clinical examination: A scoping review

Background: The objective structured clinical examination (OSCE), originally designed with experts assessing trainees’ competence, is more frequently employed with an element of peer assessment and feedback. Although peer assessment in higher education has been studied, its role in OSCEs has not reviewed.

Aims: The aim of this study is to conduct a scoping review and explore the role of peer assessment and feedback in the OSCE.

Methods: Electronic database and hand searching yielded 507 articles. Twenty-one full records were screened, of which 13 were included in the review. Two independent reviewers completed each step of the review.

Results: Peer-based OSCEs are used to assess students’ accuracy in assessing OSCE performance and to promote learning. Peer examiners (PE) tend to award better global ratings and variable checklist ratings compared to faculty and provide high-quality feedback. Participating in these OSCEs is perceived as beneficial for learning.

Conclusions: Peer assessment and feedback can be used to gauge PE reliability and promote learning. Teachers using these OSCEs must use methodology which fits their purpose. Competency-based education calls for diversification of assessment practices and asks how assessment impacts learning; the peer-based OSCE responds to these demands and will become an important practice in health professions education.     

How characteristic routines of clinical departments influence students’ self-regulated learning: A grounded theory study

Background: In clerkships, students are expected to self-regulate their learning. How clinical departments and their routine approach on clerkships influences students’ self-regulated learning (SRL) is unknown.

Aim: This study explores how characteristic routines of clinical departments influence medical students’ SRL.

Methods: Six focus groups including 39 purposively sampled participants from one Dutch university were organized to study how characteristic routines of clinical departments influenced medical students’ SRL from a constructivist paradigm, using grounded theory methodology. The focus groups were audio recorded, transcribed verbatim and were analyzed iteratively using constant comparison and open, axial and interpretive coding.

Results: Students described that clinical departments influenced their SRL through routines which affected the professional relationships they could engage in and affected their perception of a department’s invested effort in them. Students’ SRL in a clerkship can be supported by enabling them to engage others in their SRL and by having them feel that effort is invested in their learning.

Conclusions: Our study gives a practical insight in how clinical departments influenced students’ SRL. Clinical departments can affect students’ motivation to engage in SRL, influence the variety of SRL strategies that students can use and how meaningful students perceive their SRL experiences to be.     

Twelve tips for medical students to make the best use of ward-based learning

Background: With a multitude of healthcare professionals willing to teach, a placement on a ward is an invaluable learning opportunity; yet as students, we often struggle to maximize this placements’ potential.

Aim: This article provides 12 tips for medical students to optimize their learning in a ward-based environment.

Methods: Current literature and personal experiences of the authors were used to develop the tips.

Results: The 12 tips are (1) prepare yourself, (2) identify knowledge, skills, and attributes, (3) engage in peer-to-peer learning, (4) get to know the interprofessional team, (5) talk to the patient first, (6) present findings and gain feedback, (7) tap into seniors’ experience, (8) immerse yourself and be proactive, (9) check patient notes, (10) manage the clock, (11) enhance your CV, and (12) embrace the spirit of lifelong learning.

Conclusion: These tips will enable us to effectively improve our learning and positively shape us into outstanding future doctors.     

Comparing a longitudinal integrated clerkship with traditional hospital-based rotations in a rural setting

Context: Longitudinal integrated clerkships (LIC) are widely used as an educational method, particularly in rural areas. They are good for facilitating hands-on learning and deep relationships between student, patients, and supervisors.

Objectives: This study sought to examine and compare learning experience of third-year rural medical students studying specialties (women’s health, aged care, child and adolescent heath, mental health, general practice) by either a traditional hospital-based rotation or a LIC in a rural general practice setting.

Methods: Data was collected from two groups of rural students (LIC; traditional hospital-based) over two academic years, utilizing focus groups to investigate general experiences of living and learning rurally, within the different educational models.

Results: Results reaffirmed that there was no perceived academic disadvantage to studying medicine rurally. Studying medicine in a rural area provides increased access to patients, more hands-on experience, and close relationships with patients and colleagues. LIC students reported increased confidence in clinical skills, felt better prepared for internship, however experienced more social isolation than students in hospital-based rotations.

Conclusions: Students undergoing a rural LIC feel more confident in their clinical skills and preparedness for practice than other rural students. This study supports the use of LICs as a powerful educational tool.     

Effective methods of teaching and learning in anatomy as a basic science: A BEME systematic review: BEME guide no. 44

Background: Anatomy is a subject essential to medical practice, yet time committed to teaching is on the decline, and resources required to teach anatomy is costly, particularly dissection. Advances in technology are a potential solution to the problem, while maintaining the quality of teaching required for eventual clinical application.

Aim: To identify methods used to teach anatomy, including those demonstrated to enhance knowledge acquisition and retention.

Methods: PubMed, CINAHL, ERIC, Academic OneFile, ProQuest, SAGE journals and Scopus were search from the earliest entry of each database to 31 August 2015. All included articles were assessed for methodological quality and low quality articles were excluded from the study. Studies were evaluated by assessment scores, qualitative outcomes where included as well as a modified Kirkpatrick model.

Results: A total of 17,820 articles were initially identified, with 29 included in the review. The review found a wide variety of teaching interventions represented in the range of studies, with CAI/CAL studies predominating in terms of teaching interventions, followed by simulation. In addition to this, CAI/CAL and simulation studies demonstrated better results overall compared to traditional teaching methods and there is evidence to support CAI/CAL as a partial replacement for dissection or a valuable tool in conjunction with dissection.

Conclusions: This review provides evidence in support of the use of alternatives to traditional teaching methods in anatomy, in particular, the use of CAI/CAL with a number of high quality, low risk of bias studies supporting this.     

Utility of selection methods for specialist medical training: A BEME (best evidence medical education) systematic review: BEME guide no. 45

Background: Selection into specialty training is a high-stakes and resource-intensive process. While substantial literature exists on selection into medical schools, and there are individual studies in postgraduate settings, there seems to be paucity of evidence concerning selection systems and the utility of selection tools in postgraduate training environments.

Aim: To explore, analyze and synthesize the evidence related to selection into postgraduate medical specialty training.

Method: Core bibliographic databases including PubMed; Ovid Medline; Embase, CINAHL; ERIC and PsycINFO were searched, and a total of 2640 abstracts were retrieved. After removing duplicates and screening against the inclusion criteria, 202 full papers were coded, of which 116 were included.

Results: Gaps in underlying selection frameworks were illuminated. Frameworks defined by locally derived selection criteria, and heavily weighed on academic parameters seem to be giving way to the evidencing of competency-based selection approaches in some settings.

Regarding selection tools, we found favorable psychometric evidence for multiple mini-interviews, situational judgment tests and clinical problem-solving tests, although the bulk of evidence was mostly limited to the United Kingdom. The evidence around the robustness of curriculum vitae, letters of recommendation and personal statements was equivocal. The findings on the predictors of past performance were limited to academic criteria with paucity of long-term evaluations. The evidence around nonacademic criteria was inadequate to make an informed judgment.

Conclusions: While much has been gained in understanding the utility of individual selection methods, though the evidence around many of them is equivocal, the underlying theoretical and conceptual frameworks for designing holistic and equitable selection systems are yet to be developed.     

Peer group reflection on student ratings stimulates clinical teachers to generate plans to improve their teaching

Background: Previous studies have demonstrated that student ratings of a teachers’ performance do not incentivize clinical teachers to reflect critically and generate plans to improve their teaching. Peer group reflection might offer a solution in mediating this change.

Aim: To investigate: (a) to which extent clinical teachers perceive self-evaluation, student ratings and peer group reflection effective; and (b) whether additional peer group reflection fosters critical reflection and the translation of feedback into concrete plans of action.

Method: We conducted a quasi-experiment, inviting two groups of 10 clinical teachers each (1) to complete a self-evaluation and (2) subsequently examine their student ratings. One group participated in (3) an additional peer group reflection meeting. All participants were finally requested to define plans for improvement and evaluate each activity’s effectiveness.

Results: Participants perceived all three activities to be effective. Levels of reflection did not differ across the two groups. However, participation in peer group reflection did result in generating more concrete plans to change clinical teaching.

Conclusions: Peer group reflection on student ratings shows promise as tool to assist clinical teachers in generating plans for improvement. Future research should focus on whether teaching indeed improves with the introduction of peer group reflection.     

An exploration of teaching presence in online interprofessional education facilitation

Background: Although the prevalence of online asynchronous interprofessional education (IPE) has increased in the last decade, little is known about the processes of facilitation in this environment. The teaching presence element of the Community of Inquiry Framework offers an approach to analyze the contributions of online facilitators, however, to date it has only been used on a limited basis in health professions education literature.

Aim: Using an exploratory case study design, we explored the types of contributions made by IPE facilitators to asynchronous interprofessional team discussions by applying the notion of teaching presence.

Methods: Using a purposeful sampling approach, we analyzed 14 facilitators’ contributions to asynchronous team discussion boards in an online IPE course. We analyzed data using directed content analysis based on the key indicators of teaching presence.

Results: The online IPE facilitators undertook the three critical pedagogical functions identified in teaching presence: facilitating discourse, direct instruction, and instructional design and organization. While our data fitted well with a number of key activities embedded in these three functions, further modification of the teaching presence concept was needed to describe our facilitators’ teaching presence.

Conclusions: This study provides an initial insight into the key elements of online asynchronous IPE facilitation. Further research is required to continue to illuminate the complexity of online asynchronous IPE facilitation.     

Benchmarking for research-related competencies – a curricular mapping approach at medical faculties in Germany

Objectives: Internationally, scientific and research-related competencies need to be sufficiently targeted as core outcomes in many undergraduate medical curricula. Since 2015, standards have been recommended for Germany in the National Competency-based Learning Objective Catalogue in Medicine (NKLM). The aim of this study is to develop a multi-center mapping approach for curricular benchmarking against national standards and against other medical faculties.

Method: A total of 277 faculty members from four German medical faculties have mapped the local curriculum against the scientific and research-related NKLM objectives, using consented procedures, metrics, and tools. The amount of mapping citations of each objective is used as indicator for its weighting in the local curriculum. Achieved competency levels after five-year education are compared.

Results: All four programs fulfill the NKLM standards, with each emphasizing different sub-competencies explicitly in writing (Scholar: 17–41% of all courses; Medical Scientific Skills: 14–37% of all courses). Faculties show major or full agreement in objective weighting: Scholar 44%, scientific skills 79%. The given NKLM competency level is met or even outperformed in 78–100% of the courses.

Conclusions: The multi-center mapping approach provides an informative dataset allowing curricular diagnosis by external benchmarking and guidance for optimization of local curricula.     

International infectious diseases teaching to undergraduate medical students: A successful European collaborative experience*

Context: The emerging global-health paradigm requires medical teaching to be continuously redefined and updated; to this end, transnational approaches should be encouraged and medical training harmonized. Infectious diseases (ID) teaching in the current context of emerging infections, fast-increasing bacterial resistance and large-scale human migration, was chosen to develop a common international course.

Objective: We report the successful implementation of a joint European undergraduate course aiming to (i) develop a common ID core curriculum among European medical schools; (ii) promote mobility among teachers and students (iii) promote international cooperation among European teachers.

Methods: The course was built around teachers’ mobility. It was delivered in English by a team of European medical educators from Paris Descartes University, Università Cattolica del Sacro Cuore in Rome and the University of Edinburgh to groups of 25–30 undergraduate medical students at each university. Partner Institutions officially recognized the course as substitutive of or additive to the regular curriculum.

Results: The course has been running for 3 years and received excellent satisfaction scores by students and staff as regards to scientific content, pedagogy and international exchanges.

Conclusion: This cooperative approach demonstrates the feasibility of a harmonized European undergraduate medical education, having ID as a test experiment for future developments.     

Twelve tips for effective body language for medical educators

Background: A significant proportion of human communication is nonverbal. Although the fields of business and psychology have significant literature on effectively using body language in a variety of situations, there is limited literature on effective body language for medical educators.

Aim: To provide 12 tips to highlight effective body language strategies and techniques for medical educators.

Method: The tips provided are based on our experiences and reflections as clinician-educators and the available literature.

Results: The 12 tips presented offer specific strategies to engage learners, balance learner participation, and bring energy and passion to teaching.

Conclusions: Medical educators seeking to maximize their effectiveness would benefit from an understanding of how body language affects a learning environment and how body language techniques can be used to engage audiences, maintain attention, control challenging learners, and convey passion for a topic. Understanding and using body language effectively is an important instructional skill.     

Why do surgeons teach? A qualitative analysis of motivation in excellent surgical educators

Introduction: Given that teaching is so vital to the maintenance of the medical profession, it is surprising that few authors have examined the factors which motivate physicians and surgeons to engage in this activity.

Aim: It was the aim of this study to examine the factors which motivate excellent surgical educators to teach.

Methodology: Grounded theory methodology was used to analyze transcribed semi-structured interviews. The top 20 ranked surgical educators at the University of Alberta were invited to participate. In total, 15 surgeons of various specialties were interviewed.

Results: There were five main factors which motivate surgeons to teach. These were: (1) a sense of responsibility to teach future physicians (2) an intrinsic enjoyment of teaching (3) the need to maintain and expand one’s own knowledge base (4) watching students develop into competent practicing physicians and playing a role in their success, and (5) fostering positive lifelong professional relationships with learners.

Discussion: This is the first study in surgery to use an explorative qualitative methodology. This framework of motivating factors can be used to guide professional development activities. The framework represents motivating factors for those known to be effective teachers only. We plan to use this information to engage more surgeons in teaching by providing a basis for promoting education as part of a busy surgical practice.     

Social media and professional identity: Pitfalls and potential

Introduction: Social media developments have completely changed how information is accessed and communicated. While great potential exists with these platforms, recent reports of online unprofessional behavior by doctors has threatened the medical professional identity; a matter of critical importance for clinicians and medical educators. This paper outlines a role for social media in facilitating support for clinicians and medical teachers; it will raise awareness of pitfalls and explain ethical and legal guidelines.

Methods: An analysis of inappropriate behaviors and conflicting attitudes regarding what is acceptable in online posts, including the inter-generational contrast in online presence and perceptions of where the boundaries lie.

Results and discussion: Guidance documents are analyzed and potentially confusing and conflicting statements are identified and clarified. The authors believe that clinicians and medical students must follow ethical imperatives in both personal and professional spheres.

Conclusions: It is essential that medical educational and professional bodies encourage clinicians to support one another and share information online while providing clear legal and ethical advice on maintaining standards and avoiding common pitfalls. Education on the responsible use of social media and associated risk awareness should be a priority for medical school curricula.     

Creating a test blueprint for a progress testing program: A paired-comparisons approach

Context: Creating a new testing program requires the development of a test blueprint that will determine how the items on each test form are distributed across possible content areas and practice domains. To achieve validity, categories of a blueprint are typically based on the judgments of content experts. How experts judgments are elicited and combined is important to the quality of resulting test blueprints.

Methods: Content experts in dentistry participated in a day-long faculty-wide workshop to discuss, refine, and confirm the categories and their relative weights. After reaching agreement on categories and their definitions, experts judged the relative importance between category pairs, registering their judgments anonymously using iClicker, an audience response system. Judgments were combined in two ways: a simple calculation that could be performed during the workshop and a multidimensional scaling of the judgments performed later.

Results: Content experts were able to produce a set of relative weights using this approach. The multidimensional scaling yielded a three-dimensional model with the potential to provide deeper insights into the basis of the experts’ judgments.

Conclusion: The approach developed and demonstrated in this study can be applied across academic disciplines to elicit and combine content experts judgments for the development of test blueprints.     

Developing a competency framework for academic physicians

Background: There is a mismatch between the requirements of the multifaceted role of academic physicians and their education. Medical institutions use faculty development initiatives to support their junior academic physicians, however, these rarely revolve around academic physician competencies. The aim of this study was to identify these academic physician competencies and develop a competency framework customized to an organizational context.

Methods: The authors conducted semi-structured interviews and Critical Incident Technique with 25 academic physicians at a teaching medical center in the Middle East region inquiring about the behaviors of academic physicians in teaching, clinical, research, and administrative roles.

Results: Using content analysis, the authors identified 16 competencies: five “Supporting Competencies”, common to all four roles of academic physicians, and 11 “Function-Specific Competencies”, specific to the role being fulfilled. The developed framework shared similarities with frameworks reported in the literature but also had some distinctions.

Conclusions: The framework developed represents a step towards closing the gap between the skills medical students are taught and the skills required of academic physicians. The model was customized to the context of the current organization and included a future orientation and addressed the literature calling for increasing focus on the administrative skills of academic physicians.     

A systematic review of the published literature on team-based learning in health professions education

Purpose: Summarize the published literature on team-based learning (TBL) in health professions education (HPE) using the TBL conceptual framework to identify gaps that can guide future research

Methods: PubMed, Web of Science, ERIC, and Google Scholar were searched through May 2016 for English-language articles regarding the use of TBL in HPE. Reviewers independently extracted data and coded for the seven elements in Michaelsen’s Model of TBL.

Results: A total of 118 articles met inclusion criteria. The number of articles published yearly on TBL has grown steadily, more than tripling between 2011 and 2016. Most studies (55; 47%) involved undergraduate medical students and took place in the US (72; 61%). The most commonly studied framework component was Teacher and Learner Attitudes (97; 82%). Other commonly studied elements included Learning Outcomes (85; 72%) and Team Characteristics (25; 21%). Contextual Factors affecting TBL was addressed in one study.

Conclusions: A substantial body of literature examines the effect that TBL has on traditional measures of achievement. However, many dimensions of TBL have not been well studied, including Teacher Decisions about TBL, Contextual Factors that affect TBL, Learners’ Engagement, and Pattern of Engagement within Teams. Future research in these areas could determine the best use of TBL in HPE.     

Uncovering the unknown: A grounded theory study exploring the impact of self-awareness on the culture of feedback in residency education

Aim: Self-assessment and reflection are essential for meaningful feedback. We aimed to explore whether the well-known Johari window model of self-awareness could guide feedback conversations between faculty and residents and enhance the institutional feedback culture.

Methods: We had previously explored perceptions of residents and faculty regarding sociocultural factors impacting feedback. We re-analyzed data targeting themes related to self-assessment, reflection, feedback seeking and acceptance, aiming to generate individual and institutional feedback strategies applicable to each quadrant of the window.

Results: We identified the following themes for each quadrant: (1) Behaviors known to self and others – Validating the known; (2) Behaviors unknown to self but known to others – Accepting the blind; (3) Behaviors known to self and unknown to others – Disclosure of hidden; and (4) Behaviors unknown to self and others – Uncovering the unknown. Normalizing self-disclosure of limitations, encouraging feedback seeking, training in nonjudgmental feedback and providing opportunities for longitudinal relationships could promote self-awareness, ultimately expanding the “open” quadrant of the Johari window.

Conclusions: The Johari window, a model of self-awareness in interpersonal communications, could provide a robust framework for individuals to improve their feedback conversations and institutions to design feedback initiatives that enhance its quality and impact.