Peer assessment in the objective structured clinical examination: A scoping review

Background: The objective structured clinical examination (OSCE), originally designed with experts assessing trainees’ competence, is more frequently employed with an element of peer assessment and feedback. Although peer assessment in higher education has been studied, its role in OSCEs has not reviewed.

Aims: The aim of this study is to conduct a scoping review and explore the role of peer assessment and feedback in the OSCE.

Methods: Electronic database and hand searching yielded 507 articles. Twenty-one full records were screened, of which 13 were included in the review. Two independent reviewers completed each step of the review.

Results: Peer-based OSCEs are used to assess students’ accuracy in assessing OSCE performance and to promote learning. Peer examiners (PE) tend to award better global ratings and variable checklist ratings compared to faculty and provide high-quality feedback. Participating in these OSCEs is perceived as beneficial for learning.

Conclusions: Peer assessment and feedback can be used to gauge PE reliability and promote learning. Teachers using these OSCEs must use methodology which fits their purpose. Competency-based education calls for diversification of assessment practices and asks how assessment impacts learning; the peer-based OSCE responds to these demands and will become an important practice in health professions education.     

Ward round simulation in final year medical students: Does it promote students learning?

Introduction: Ward round skills are essential for doctors in hospital settings. Literature shows medical students’ deficiencies in these skills. Simulation has been used to train these skills. However, exposing learners to simulation at an early stage may be associated with a high cognitive load and limited learning. This study aims to determine how students experience this load and its interplay with performance and which factors promote and impair learning.

Methods: Fifty-six final year medical students participated in a simulated ward round training exercise. Both students’ performance and cognitive load were measured to determine if there was any correlation and interviews were carried out to understand which factors support and impair learning.

Results: Performance scores revealed deficiencies in ward round skills. Students experienced a cognitive load that weakly correlated with performance. Qualitative findings provided important insights into simulated ward-based learning. It is clear that well-designed clinical scenarios, prioritization tasks, teamwork and feedback support students’ learning process whereas distractions impair learning.

Conclusions: WRS proved to be a good teaching method to improve clinical skills at this stage as the cognitive load is not too high to impair learning. Hence, including tasks in the simulation design can enhance the learning process.     

Eye-tracking technology in medical education: A systematic review

Background: Eye-tracking technology is an established research tool within allied industries such as advertising, psychology and aerospace. This review aims to consolidate literature describing the evidence for use of eye-tracking as an adjunct to traditional teaching methods in medical education.

Methods: A systematic literature review was conducted in line with STORIES guidelines. A search of EMBASE, OVID MEDLINE, PsycINFO, TRIP database, and Science Direct was conducted until January 2017. Studies describing the use of eye-tracking in the training, assessment, and feedback of clinicians were included in the review.

Results: Thirty-three studies were included in the final qualitative synthesis. Three studies were based on the use of gaze training, three studies on the changes in gaze behavior during the learning curve, 17 studies on clinical assessment and six studies focused on the use of eye-tracking methodology as a feedback tool. The studies demonstrated feasibility and validity in the use of eye-tracking as a training and assessment method.

Conclusions: Overall, eye-tracking methodology has contributed significantly to the training, assessment, and feedback practices used in the clinical setting. The technology provides reliable quantitative data, which can be interpreted to give an indication of clinical skill, provide training solutions and aid in feedback and reflection. This review provides a detailed summary of evidence relating to eye-tracking methodology and its uses as a training method, changes in visual gaze behavior during the learning curve, eye-tracking methodology for proficiency assessment and its uses as a feedback tool.     

Comparing a longitudinal integrated clerkship with traditional hospital-based rotations in a rural setting

Context: Longitudinal integrated clerkships (LIC) are widely used as an educational method, particularly in rural areas. They are good for facilitating hands-on learning and deep relationships between student, patients, and supervisors.

Objectives: This study sought to examine and compare learning experience of third-year rural medical students studying specialties (women’s health, aged care, child and adolescent heath, mental health, general practice) by either a traditional hospital-based rotation or a LIC in a rural general practice setting.

Methods: Data was collected from two groups of rural students (LIC; traditional hospital-based) over two academic years, utilizing focus groups to investigate general experiences of living and learning rurally, within the different educational models.

Results: Results reaffirmed that there was no perceived academic disadvantage to studying medicine rurally. Studying medicine in a rural area provides increased access to patients, more hands-on experience, and close relationships with patients and colleagues. LIC students reported increased confidence in clinical skills, felt better prepared for internship, however experienced more social isolation than students in hospital-based rotations.

Conclusions: Students undergoing a rural LIC feel more confident in their clinical skills and preparedness for practice than other rural students. This study supports the use of LICs as a powerful educational tool.     

Twelve tips for medical students to make the best use of ward-based learning

Background: With a multitude of healthcare professionals willing to teach, a placement on a ward is an invaluable learning opportunity; yet as students, we often struggle to maximize this placements’ potential.

Aim: This article provides 12 tips for medical students to optimize their learning in a ward-based environment.

Methods: Current literature and personal experiences of the authors were used to develop the tips.

Results: The 12 tips are (1) prepare yourself, (2) identify knowledge, skills, and attributes, (3) engage in peer-to-peer learning, (4) get to know the interprofessional team, (5) talk to the patient first, (6) present findings and gain feedback, (7) tap into seniors’ experience, (8) immerse yourself and be proactive, (9) check patient notes, (10) manage the clock, (11) enhance your CV, and (12) embrace the spirit of lifelong learning.

Conclusion: These tips will enable us to effectively improve our learning and positively shape us into outstanding future doctors.     

Medical education of attention: A qualitative study of learning to listen to sound

Introduction: There has been little qualitative research examining how physical examination skills are learned, particularly the sensory and subjective aspects of learning. The authors set out to study how medical students are taught and learn the skills of listening to sound.

Methods: As part of an ethnographic study in Melbourne, 15 semi-structured in-depth interviews were conducted with students and teachers as a way to reflect explicitly on their learning and teaching.

Results: From these interviews, we found that learning the skills of listening to lung sounds was frequently difficult for students, with many experiencing awkwardness, uncertainty, pressure, and intimidation. However not everyone found this process difficult. Often those who had studied music reported finding it easier to be attentive to the frequency and rhythm of body sounds and find ways to describe them.

Conclusions: By incorporating, distinctively in medical education, theoretical insights into “attentiveness” from anthropology and science and technology studies, the article suggests that musical education provides medical students with skills in sensory awareness. Training the senses is a critical aspect of diagnosis that needs to be better addressed in medical education. Practical approaches for improving students’ education of attention are proposed.     

The benefits of sequential testing: Improved diagnostic accuracy and better outcomes for failing students

Introduction: In recent decades, there has been a move towards standardized models of assessment where all students sit the same test (e.g. OSCE). By contrast, in a sequential test the examination is in two parts, a “screening” test (S1) that all candidates take, and then a second “test” (S2) which only the weaker candidates sit. This article investigates the diagnostic accuracy of this assessment design, and investigates failing students’ subsequent performance under this model.

Methods: Using recent undergraduate knowledge and performance data, we compare S1 “decisions” to S2 overall pass/fail decisions to assess diagnostic accuracy in a sequential model. We also evaluate the longitudinal performance of failing students using changes in percentile ranks over a full repeated year.

Findings: We find a small but important improvement in diagnostic accuracy under a sequential model (of the order 2–4% of students misclassified under a traditional model). Further, after a resit year, weaker students’ rankings relative to their peers improve by 20–30 percentile points.

Discussion: These findings provide strong empirical support for the theoretical arguments in favor of a sequential testing model of assessment, particularly that diagnostic accuracy and longitudinal assessment outcomes post-remediation for the weakest students are both improved.     

How physicians teach in the clinical setting: The embedded roles of teaching and clinical care

Background: Clinical teaching lies at the heart of medical education. However, few studies have explored the embedded nature of teaching and clinical care. The goal of this study was to examine the process of clinical teaching as it naturally, and spontaneously, unfolds in a broad range of authentic contexts with medical students and residents.

Methods: This focused ethnographic study consisted of 160 hours of participant observation and field interviews with three internal medicine teams. Thematic analysis guided data organization and interpretation.

Findings: Three overlapping themes emerged: the interconnectedness between clinical work and pedagogy; a multiplicity of teachers; and the influence of space and artifacts on teaching and learning. Clinical teaching, which was deeply embedded in clinical care, was influenced by the acuity of patient problems, learner needs, and the context in which teaching unfolded; it also occurred on a spectrum that included planned, opportunistic, formal, and informal teaching (and learning).

Conclusions: Study findings suggest that clinical teaching, which is marked by an intersection between service and teaching, can be viewed as an example of work-based teaching. They also yield suggestions for the enhancement of clinical teaching in inpatient settings, faculty development, and educational policies that recognize clinical teaching and learning.     

How characteristic routines of clinical departments influence students’ self-regulated learning: A grounded theory study

Background: In clerkships, students are expected to self-regulate their learning. How clinical departments and their routine approach on clerkships influences students’ self-regulated learning (SRL) is unknown.

Aim: This study explores how characteristic routines of clinical departments influence medical students’ SRL.

Methods: Six focus groups including 39 purposively sampled participants from one Dutch university were organized to study how characteristic routines of clinical departments influenced medical students’ SRL from a constructivist paradigm, using grounded theory methodology. The focus groups were audio recorded, transcribed verbatim and were analyzed iteratively using constant comparison and open, axial and interpretive coding.

Results: Students described that clinical departments influenced their SRL through routines which affected the professional relationships they could engage in and affected their perception of a department’s invested effort in them. Students’ SRL in a clerkship can be supported by enabling them to engage others in their SRL and by having them feel that effort is invested in their learning.

Conclusions: Our study gives a practical insight in how clinical departments influenced students’ SRL. Clinical departments can affect students’ motivation to engage in SRL, influence the variety of SRL strategies that students can use and how meaningful students perceive their SRL experiences to be.     

How teachers can help learners build storage and retrieval strength

Aim: To be an effective teacher, content expertise is necessary but alone does not guarantee optimal learning outcomes for students. In this article, the authors discuss ways in which medical teachers can shape the learning of their students and enable them to become more efficient and effective learners.

Methods: Using Bjork and Bjork's new theory of disuse as their framework, the authors discuss strategies to improve storage strength of to-be-learned information and strategies to improve retrieval strength of learned information.

Results: Strategies to improve storage strength include optimizing cognitive load, providing causal explanations, and giving effective feedback. Strategies to improve retrieval strength include situated cognition and various types of retrieval practice.

Conclusions: Adopting these teaching strategies should hopefully help teachers improve the learning outcomes of their students, but there is still a need for further research into the science of learning and the science of instruction, including comparative effectiveness of different teaching strategies and how best to translate findings from the psychology literature into medical education.     

Student academic performance in rural clinical schools: The impact of cohort size and competition

Objectives: The Deakin University School of Medicine commenced in 2008 as a rurally focused medical school in south-eastern Australia. This research was designed to examine the effectiveness of the school’s adoption of small regional clinical school settings.

Methods: A retrospective cohort study of the first two cohorts of students was employed to assess academic performance at each of five geographically dispersed clinical training sites, with varying student cohort sizes. The Dundee Ready Education Environment Measure (DREEM) questionnaire provided quantitative data regarding the students’ perception of their educational environment. The data were analyzed using univariate and multivariate analyses.

Results: The highest examination scores, and greatest satisfaction with educational environment, were associated with the clinical school that had a small-sized group of students and was not co-located with another medical school. These differences remained after adjusting for multiple potential confounding factors.

Conclusion: The smaller sites appear to have provided superior support for student learning in this new medical school. This advantage diminishes when smaller cohorts are co-located with students from other medical schools. Cohort size and co-location of medical school curricula may be important independent variables for researchers to consider when comparing the results of clinical education innovations in different settings.     

Twelve tips on how to establish a new undergraduate firm on a critical care unit

Background: Little is known about undergraduate teaching in critical care unit (CrCU) and many undergraduate curricula lack placements in CrCU.

Aims: To describe how our CrCU succeeded in developing a novel placement for Year 3 undergraduate medical students.

Methods: Particular emphasis was placed on a robust timetable incorporating a variety of activities, a dedicated and thorough induction, and a mix of teaching methods such as formal and informal, consultant-led, and skills. Services allied to CrCU were also utilized.

Results: Our new firm has exceeded all expectations and, based on student feedback, received the “Firm of the Year” award for several years in succession. It now serves as a model of undergraduate teaching in our hospital.

Conclusions: Educationalists and intensivists should work together to unlock the full potential of this rich learning environment. Professional societies in critical care medicine should take the opportunity to develop more interest in undergraduate medical education.     

Feedback mapping – The curricular cornerstone of an “educational alliance”

Purpose: The “educational alliance” concept articulates a collaborative framework to facilitate effective feedback through transparency of opportunities aligned with learner stage and intended educational outcomes. Using this framework, we evaluated feedback across a 5 year undergraduate medical program to support embedding a successful learner – teacher “educational alliance”.

Method: A comprehensive mapping exercise used an iterative action research process of source documentary analysis, consultations with key curriculum stakeholders and qualitative analysis. The “educational alliance” model provided a critical lens through which to ensure feedback opportunities aligned with intended learning outcomes and developmental progression.

Results: Key information about the type, frequency and timing of feedback opportunities was identified within 188 curricular components. The purpose and intended learning outcomes were mapped, aligning with the stage of learning and with an expectation of supporting learners’ capacity for self-regulation. This focus providing clear articulation of feedback opportunities supported the longitudinal developmental curricular review, and facilitated enhanced awareness of dialogic feedback within the “educational alliance”.

Conclusions: Explicit alignment of learning intentions between learner and educator is key to forming a successful “educational alliance”. The feedback map provides clarity ensuring mutual understanding of intended learning outcomes. The iterative process additionally certified feedback aligned with maturing learner developmental needs across the program.     

Defining a set of common interprofessional learning competencies for health profession students

Introduction: Increasingly recognized as a core component of contemporary health profession education, interprofessional learning outcomes remain difficult to define and assess across disciplines. The aim of this study was to identify a single set of interprofessional learning competency statements with relevance to all health professions.

Methods and results: Six national and international interprofessional competency frameworks were reviewed and combined to give a total of 165 competency statements. Following a process of mapping and grouping these statements into common content areas, duplicate content was removed. In addition, content deemed as a core competency for one or more individual health professions was removed. A round table of experts reviewed the remaining statements and agreed a final set of eight. Each statement was expressed as a specific learning outcome that could be assessed and which described behaviors and practices that students could routinely expect to engage with, and participate in, during the course of their study.

Conclusion: Identifying specific interprofessional competencies that students of all health professions require will enable more effective implementation of interprofessional learning activities and assessment within the core curriculum.     

Peer group reflection on student ratings stimulates clinical teachers to generate plans to improve their teaching

Background: Previous studies have demonstrated that student ratings of a teachers’ performance do not incentivize clinical teachers to reflect critically and generate plans to improve their teaching. Peer group reflection might offer a solution in mediating this change.

Aim: To investigate: (a) to which extent clinical teachers perceive self-evaluation, student ratings and peer group reflection effective; and (b) whether additional peer group reflection fosters critical reflection and the translation of feedback into concrete plans of action.

Method: We conducted a quasi-experiment, inviting two groups of 10 clinical teachers each (1) to complete a self-evaluation and (2) subsequently examine their student ratings. One group participated in (3) an additional peer group reflection meeting. All participants were finally requested to define plans for improvement and evaluate each activity’s effectiveness.

Results: Participants perceived all three activities to be effective. Levels of reflection did not differ across the two groups. However, participation in peer group reflection did result in generating more concrete plans to change clinical teaching.

Conclusions: Peer group reflection on student ratings shows promise as tool to assist clinical teachers in generating plans for improvement. Future research should focus on whether teaching indeed improves with the introduction of peer group reflection.     

The effectiveness of sequencing virtual patients with lectures in a deductive or inductive learning approach

Background: Virtual patients (VPs) can be sequenced with other instructional methods in different ways.

Aim: To investigate the effect of sequencing VPs with lectures in a deductive approach, in comparison with an inductive approach, on students’ knowledge acquisition, retention, and transfer.

Methods: For two different topics, 84 out of 87 students have participated in the lecture and VP sessions. Students from female and male campuses have been randomly assigned to one of the two learning approaches (deductive and inductive), yielding four experimental groups. Each group received a lecture session and an independent VP learning activity, which either followed the lecture session in the deductive group or preceded it in the inductive group. Students were administrated knowledge acquisition and retention written tests as well as transfer tests using two new VPs.

Results: There was no significant effect for the learning approach on knowledge acquisition or retention, while for knowledge transfer, males have benefited from the inductive approach in topic 1 while in the more complex topic 2, they have benefited from the deductive approach. On the other hand, females seem to be largely unaffected by learning approach.

Conclusions: Sequencing VPs in inductive and deductive learning approaches leads to no significant differences on students’ performance when full guidance is offered in the inductive approach.     

Why do surgeons teach? A qualitative analysis of motivation in excellent surgical educators

Introduction: Given that teaching is so vital to the maintenance of the medical profession, it is surprising that few authors have examined the factors which motivate physicians and surgeons to engage in this activity.

Aim: It was the aim of this study to examine the factors which motivate excellent surgical educators to teach.

Methodology: Grounded theory methodology was used to analyze transcribed semi-structured interviews. The top 20 ranked surgical educators at the University of Alberta were invited to participate. In total, 15 surgeons of various specialties were interviewed.

Results: There were five main factors which motivate surgeons to teach. These were: (1) a sense of responsibility to teach future physicians (2) an intrinsic enjoyment of teaching (3) the need to maintain and expand one’s own knowledge base (4) watching students develop into competent practicing physicians and playing a role in their success, and (5) fostering positive lifelong professional relationships with learners.

Discussion: This is the first study in surgery to use an explorative qualitative methodology. This framework of motivating factors can be used to guide professional development activities. The framework represents motivating factors for those known to be effective teachers only. We plan to use this information to engage more surgeons in teaching by providing a basis for promoting education as part of a busy surgical practice.     

Challenges of feedback provision in the workplace: A qualitative study of emergency medicine residents and teachers

Background: Feedback is an effective pedagogical tool in clinical teaching and learning, but is often perceived as unsatisfactory. Little is known about the effect of a busy clinical environment on feedback-giving and -seeking behaviors. This study aims to determine the perceptions and challenges of feedback provision in a busy clinical setting, exemplified by an emergency department (ED).

Methods: A qualitative semi-structured interview study design was employed. Thirty-six participants (18 attending physicians, 18 residents) were purposively sampled from three EDs in northern Taiwan between August 2015 and July 2016. Interviews were recorded, transcribed, and analyzed thematically.

Results: Three major themes were identified with illustrative quotes: (1) the balance between patient safety and providing feedback, (2) variability in feedback, and (3) influential factors, barriers and enablers.

Conclusions: In real practice, clinical duties competed with the impulse to provide feedback. The variety and complexity of feedback extended beyond style and content. Clinical and contextual factors – some of which may be presented as barriers – influenced how, when and whether a teacher or learner decided to give or seek feedback.     

Twelve tips for teaching brief motivational interviewing to medical students

Background: Shifting from paternalistic to patient-centred doctor-patient relationships has seen a growing number of medical programs incorporate brief motivational interviewing training in their curriculum. Some medical educators, however, are unsure of precisely what, when, and how to incorporate such training.

Aims: This article provides educators with 12 tips for teaching brief motivational interviewing to medical students, premised on evidence-based pedagogy.

Methods: Tips were drawn from the literature and authors’ own experiences.

Results: The 12 tips are: (1) Set clear learning objectives, (2) Select experienced educators, (3) Provide theoretical perspectives, (4) Share the evidence base, (5) Outline the “spirit”, principles, and sequence, (6) Show students what it looks like, (7) Give students a scaffold to follow, (8) Provide opportunities for skill practice, (9) Involve clinical students in teaching, (10) Use varied formative and summative assessments, (11) Integrate and maintain, and (12) Reflect and evaluate.

Conclusions: We describe what to include and why, and outline when and how to teach the essential components of brief motivational interviewing knowledge and skills in a medical curriculum.