Using children as standardized patients in OSCE in pediatric occupational therapy

Background: Objective structured clinical examinations (OSCEs) have been administered in physical and psychiatric occupational therapy (OT) education, but not in pediatric OT education. The objectives were to examine the satisfaction and the influences of OSCE in pediatric OT on all participants.

Methods: The OSCE contained evaluation, intervention, and parent education stations. Sixty examinees, 44 child standardized patients (SPs), 44 chaperones, three playroom managers, 14 OSCE assistants, and 15 examiners participated in the OSCE. An OSCE video and three playrooms were prepared for child SPs.

Results: Ninety percent of the child SPs liked taking part in the OSCE and 75–85% expressed interest in participating in an OSCE the following year. Their parents appreciated the chaperones accompanying their children and giving them a memorable day. 88.3% of the examinees thought that the OSCE was helpful for their upcoming clinical training. 73.3% preferred the OSCE over the written exam. 60–93.4% considered the implementation appropriate. Most of the examiners thought that the content (80–100%) and the implementation (93.3–100%) were appropriate. Many chaperones reported having valuable experiences.

Conclusions: It is practical using child SPs in OSCE in pediatric OT. The OSCE was beneficial to all participants. It is recommended that OSCEs be included in pediatric OT education.     

A collaborative comparison of objective structured clinical examination (OSCE) standard setting methods at Australian medical schools

Background: A key issue underpinning the usefulness of the OSCE assessment to medical education is standard setting, but the majority of standard-setting methods remain challenging for performance assessment because they produce varying passing marks. Several studies have compared standard-setting methods; however, most of these studies are limited by their experimental scope, or use data on examinee performance at a single OSCE station or from a single medical school. This collaborative study between 10 Australian medical schools investigated the effect of standard-setting methods on OSCE cut scores and failure rates.

Methods: This research used 5256 examinee scores from seven shared OSCE stations to calculate cut scores and failure rates using two different compromise standard-setting methods, namely the Borderline Regression and Cohen’s methods.

Results: The results of this study indicate that Cohen’s method yields similar outcomes to the Borderline Regression method, particularly for large examinee cohort sizes. However, with lower examinee numbers on a station, the Borderline Regression method resulted in higher cut scores and larger difference margins in the failure rates.

Conclusion: Cohen’s method yields similar outcomes as the Borderline Regression method and its application for benchmarking purposes and in resource-limited settings is justifiable, particularly with large examinee numbers.     

Managing extremes of assessor judgment within the OSCE

Context: There is a growing body of research investigating assessor judgments in complex performance environments such as OSCE examinations. Post hoc analysis can be employed to identify some elements of “unwanted” assessor variance. However, the impact of individual, apparently “extreme” assessors on OSCE quality, assessment outcomes and pass/fail decisions has not been previously explored. This paper uses a range of “case studies” as examples to illustrate the impact that “extreme” examiners can have in OSCEs, and gives pragmatic suggestions to successfully alleviating problems.

Method and results: We used real OSCE assessment data from a number of examinations where at station level, a single examiner assesses student performance using a global grade and a key features checklist. Three exemplar case studies where initial post hoc analysis has indicated problematic individual assessor behavior are considered and discussed in detail, highlighting both the impact of individual examiner behavior and station design on subsequent judgments.

Conclusions: In complex assessment environments, institutions have a duty to maximize the defensibility, quality and validity of the assessment process. A key element of this involves critical analysis, through a range of approaches, of assessor judgments. However, care must be taken when assuming that apparent aberrant examiner behavior is automatically just that.     

Twelve tips for effective body language for medical educators

Background: A significant proportion of human communication is nonverbal. Although the fields of business and psychology have significant literature on effectively using body language in a variety of situations, there is limited literature on effective body language for medical educators.

Aim: To provide 12 tips to highlight effective body language strategies and techniques for medical educators.

Method: The tips provided are based on our experiences and reflections as clinician-educators and the available literature.

Results: The 12 tips presented offer specific strategies to engage learners, balance learner participation, and bring energy and passion to teaching.

Conclusions: Medical educators seeking to maximize their effectiveness would benefit from an understanding of how body language affects a learning environment and how body language techniques can be used to engage audiences, maintain attention, control challenging learners, and convey passion for a topic. Understanding and using body language effectively is an important instructional skill.     

Twelve tips on how to establish a new undergraduate firm on a critical care unit

Background: Little is known about undergraduate teaching in critical care unit (CrCU) and many undergraduate curricula lack placements in CrCU.

Aims: To describe how our CrCU succeeded in developing a novel placement for Year 3 undergraduate medical students.

Methods: Particular emphasis was placed on a robust timetable incorporating a variety of activities, a dedicated and thorough induction, and a mix of teaching methods such as formal and informal, consultant-led, and skills. Services allied to CrCU were also utilized.

Results: Our new firm has exceeded all expectations and, based on student feedback, received the “Firm of the Year” award for several years in succession. It now serves as a model of undergraduate teaching in our hospital.

Conclusions: Educationalists and intensivists should work together to unlock the full potential of this rich learning environment. Professional societies in critical care medicine should take the opportunity to develop more interest in undergraduate medical education.     

The benefits of sequential testing: Improved diagnostic accuracy and better outcomes for failing students

Introduction: In recent decades, there has been a move towards standardized models of assessment where all students sit the same test (e.g. OSCE). By contrast, in a sequential test the examination is in two parts, a “screening” test (S1) that all candidates take, and then a second “test” (S2) which only the weaker candidates sit. This article investigates the diagnostic accuracy of this assessment design, and investigates failing students’ subsequent performance under this model.

Methods: Using recent undergraduate knowledge and performance data, we compare S1 “decisions” to S2 overall pass/fail decisions to assess diagnostic accuracy in a sequential model. We also evaluate the longitudinal performance of failing students using changes in percentile ranks over a full repeated year.

Findings: We find a small but important improvement in diagnostic accuracy under a sequential model (of the order 2–4% of students misclassified under a traditional model). Further, after a resit year, weaker students’ rankings relative to their peers improve by 20–30 percentile points.

Discussion: These findings provide strong empirical support for the theoretical arguments in favor of a sequential testing model of assessment, particularly that diagnostic accuracy and longitudinal assessment outcomes post-remediation for the weakest students are both improved.     

How characteristic routines of clinical departments influence students’ self-regulated learning: A grounded theory study

Background: In clerkships, students are expected to self-regulate their learning. How clinical departments and their routine approach on clerkships influences students’ self-regulated learning (SRL) is unknown.

Aim: This study explores how characteristic routines of clinical departments influence medical students’ SRL.

Methods: Six focus groups including 39 purposively sampled participants from one Dutch university were organized to study how characteristic routines of clinical departments influenced medical students’ SRL from a constructivist paradigm, using grounded theory methodology. The focus groups were audio recorded, transcribed verbatim and were analyzed iteratively using constant comparison and open, axial and interpretive coding.

Results: Students described that clinical departments influenced their SRL through routines which affected the professional relationships they could engage in and affected their perception of a department’s invested effort in them. Students’ SRL in a clerkship can be supported by enabling them to engage others in their SRL and by having them feel that effort is invested in their learning.

Conclusions: Our study gives a practical insight in how clinical departments influenced students’ SRL. Clinical departments can affect students’ motivation to engage in SRL, influence the variety of SRL strategies that students can use and how meaningful students perceive their SRL experiences to be.     

Effective methods of teaching and learning in anatomy as a basic science: A BEME systematic review: BEME guide no. 44

Background: Anatomy is a subject essential to medical practice, yet time committed to teaching is on the decline, and resources required to teach anatomy is costly, particularly dissection. Advances in technology are a potential solution to the problem, while maintaining the quality of teaching required for eventual clinical application.

Aim: To identify methods used to teach anatomy, including those demonstrated to enhance knowledge acquisition and retention.

Methods: PubMed, CINAHL, ERIC, Academic OneFile, ProQuest, SAGE journals and Scopus were search from the earliest entry of each database to 31 August 2015. All included articles were assessed for methodological quality and low quality articles were excluded from the study. Studies were evaluated by assessment scores, qualitative outcomes where included as well as a modified Kirkpatrick model.

Results: A total of 17,820 articles were initially identified, with 29 included in the review. The review found a wide variety of teaching interventions represented in the range of studies, with CAI/CAL studies predominating in terms of teaching interventions, followed by simulation. In addition to this, CAI/CAL and simulation studies demonstrated better results overall compared to traditional teaching methods and there is evidence to support CAI/CAL as a partial replacement for dissection or a valuable tool in conjunction with dissection.

Conclusions: This review provides evidence in support of the use of alternatives to traditional teaching methods in anatomy, in particular, the use of CAI/CAL with a number of high quality, low risk of bias studies supporting this.     

Twelve tips for medical students to make the best use of ward-based learning

Background: With a multitude of healthcare professionals willing to teach, a placement on a ward is an invaluable learning opportunity; yet as students, we often struggle to maximize this placements’ potential.

Aim: This article provides 12 tips for medical students to optimize their learning in a ward-based environment.

Methods: Current literature and personal experiences of the authors were used to develop the tips.

Results: The 12 tips are (1) prepare yourself, (2) identify knowledge, skills, and attributes, (3) engage in peer-to-peer learning, (4) get to know the interprofessional team, (5) talk to the patient first, (6) present findings and gain feedback, (7) tap into seniors’ experience, (8) immerse yourself and be proactive, (9) check patient notes, (10) manage the clock, (11) enhance your CV, and (12) embrace the spirit of lifelong learning.

Conclusion: These tips will enable us to effectively improve our learning and positively shape us into outstanding future doctors.     

Twelve tips for teaching brief motivational interviewing to medical students

Background: Shifting from paternalistic to patient-centred doctor-patient relationships has seen a growing number of medical programs incorporate brief motivational interviewing training in their curriculum. Some medical educators, however, are unsure of precisely what, when, and how to incorporate such training.

Aims: This article provides educators with 12 tips for teaching brief motivational interviewing to medical students, premised on evidence-based pedagogy.

Methods: Tips were drawn from the literature and authors’ own experiences.

Results: The 12 tips are: (1) Set clear learning objectives, (2) Select experienced educators, (3) Provide theoretical perspectives, (4) Share the evidence base, (5) Outline the “spirit”, principles, and sequence, (6) Show students what it looks like, (7) Give students a scaffold to follow, (8) Provide opportunities for skill practice, (9) Involve clinical students in teaching, (10) Use varied formative and summative assessments, (11) Integrate and maintain, and (12) Reflect and evaluate.

Conclusions: We describe what to include and why, and outline when and how to teach the essential components of brief motivational interviewing knowledge and skills in a medical curriculum.     

Utility of selection methods for specialist medical training: A BEME (best evidence medical education) systematic review: BEME guide no. 45

Background: Selection into specialty training is a high-stakes and resource-intensive process. While substantial literature exists on selection into medical schools, and there are individual studies in postgraduate settings, there seems to be paucity of evidence concerning selection systems and the utility of selection tools in postgraduate training environments.

Aim: To explore, analyze and synthesize the evidence related to selection into postgraduate medical specialty training.

Method: Core bibliographic databases including PubMed; Ovid Medline; Embase, CINAHL; ERIC and PsycINFO were searched, and a total of 2640 abstracts were retrieved. After removing duplicates and screening against the inclusion criteria, 202 full papers were coded, of which 116 were included.

Results: Gaps in underlying selection frameworks were illuminated. Frameworks defined by locally derived selection criteria, and heavily weighed on academic parameters seem to be giving way to the evidencing of competency-based selection approaches in some settings.

Regarding selection tools, we found favorable psychometric evidence for multiple mini-interviews, situational judgment tests and clinical problem-solving tests, although the bulk of evidence was mostly limited to the United Kingdom. The evidence around the robustness of curriculum vitae, letters of recommendation and personal statements was equivocal. The findings on the predictors of past performance were limited to academic criteria with paucity of long-term evaluations. The evidence around nonacademic criteria was inadequate to make an informed judgment.

Conclusions: While much has been gained in understanding the utility of individual selection methods, though the evidence around many of them is equivocal, the underlying theoretical and conceptual frameworks for designing holistic and equitable selection systems are yet to be developed.     

Embedding of the progress test in an assessment program designed according to the principles of programmatic assessment

Background: Progress tests (PT) are used to assess students on topics from all medical disciplines. Progress testing is usually one of the assessment methods of the cognitive domain. There is limited knowledge on how positioning of the PT in a program of assessment (PoA) influences students’ PT scores, use of PT feedback and perceived learning value.

Methods: We compared PT total scores and use of a PT test feedback (ProF) system in two medical courses, where the PT is either used as a summative assessment or embedded in a comprehensive PoA and used formatively. In addition, an interview study was used to explore the students’ perception on use of PT feedback and perceived learning value.

Results: PT total scores were higher, with considerable effect sizes (ESs) and students made more use of ProF when the PT was embedded in a comprehensive PoA. Analysis of feedback in the portfolio stimulated students to look for patterns in PT results, link the PT to other assessment results, follow-up on learning objectives, and integrate the PT in their learning for the entire PoA.

Conclusions: Embedding the PT in an assessment program designed according to the principles of programmatic assessment positively affects PT total scores, use of PT feedback, and perceived learning value.     

An exploration of teaching presence in online interprofessional education facilitation

Background: Although the prevalence of online asynchronous interprofessional education (IPE) has increased in the last decade, little is known about the processes of facilitation in this environment. The teaching presence element of the Community of Inquiry Framework offers an approach to analyze the contributions of online facilitators, however, to date it has only been used on a limited basis in health professions education literature.

Aim: Using an exploratory case study design, we explored the types of contributions made by IPE facilitators to asynchronous interprofessional team discussions by applying the notion of teaching presence.

Methods: Using a purposeful sampling approach, we analyzed 14 facilitators’ contributions to asynchronous team discussion boards in an online IPE course. We analyzed data using directed content analysis based on the key indicators of teaching presence.

Results: The online IPE facilitators undertook the three critical pedagogical functions identified in teaching presence: facilitating discourse, direct instruction, and instructional design and organization. While our data fitted well with a number of key activities embedded in these three functions, further modification of the teaching presence concept was needed to describe our facilitators’ teaching presence.

Conclusions: This study provides an initial insight into the key elements of online asynchronous IPE facilitation. Further research is required to continue to illuminate the complexity of online asynchronous IPE facilitation.     

The effectiveness of sequencing virtual patients with lectures in a deductive or inductive learning approach

Background: Virtual patients (VPs) can be sequenced with other instructional methods in different ways.

Aim: To investigate the effect of sequencing VPs with lectures in a deductive approach, in comparison with an inductive approach, on students’ knowledge acquisition, retention, and transfer.

Methods: For two different topics, 84 out of 87 students have participated in the lecture and VP sessions. Students from female and male campuses have been randomly assigned to one of the two learning approaches (deductive and inductive), yielding four experimental groups. Each group received a lecture session and an independent VP learning activity, which either followed the lecture session in the deductive group or preceded it in the inductive group. Students were administrated knowledge acquisition and retention written tests as well as transfer tests using two new VPs.

Results: There was no significant effect for the learning approach on knowledge acquisition or retention, while for knowledge transfer, males have benefited from the inductive approach in topic 1 while in the more complex topic 2, they have benefited from the deductive approach. On the other hand, females seem to be largely unaffected by learning approach.

Conclusions: Sequencing VPs in inductive and deductive learning approaches leads to no significant differences on students’ performance when full guidance is offered in the inductive approach.     

International infectious diseases teaching to undergraduate medical students: A successful European collaborative experience*

Context: The emerging global-health paradigm requires medical teaching to be continuously redefined and updated; to this end, transnational approaches should be encouraged and medical training harmonized. Infectious diseases (ID) teaching in the current context of emerging infections, fast-increasing bacterial resistance and large-scale human migration, was chosen to develop a common international course.

Objective: We report the successful implementation of a joint European undergraduate course aiming to (i) develop a common ID core curriculum among European medical schools; (ii) promote mobility among teachers and students (iii) promote international cooperation among European teachers.

Methods: The course was built around teachers’ mobility. It was delivered in English by a team of European medical educators from Paris Descartes University, Università Cattolica del Sacro Cuore in Rome and the University of Edinburgh to groups of 25–30 undergraduate medical students at each university. Partner Institutions officially recognized the course as substitutive of or additive to the regular curriculum.

Results: The course has been running for 3 years and received excellent satisfaction scores by students and staff as regards to scientific content, pedagogy and international exchanges.

Conclusion: This cooperative approach demonstrates the feasibility of a harmonized European undergraduate medical education, having ID as a test experiment for future developments.     

Peer group reflection on student ratings stimulates clinical teachers to generate plans to improve their teaching

Background: Previous studies have demonstrated that student ratings of a teachers’ performance do not incentivize clinical teachers to reflect critically and generate plans to improve their teaching. Peer group reflection might offer a solution in mediating this change.

Aim: To investigate: (a) to which extent clinical teachers perceive self-evaluation, student ratings and peer group reflection effective; and (b) whether additional peer group reflection fosters critical reflection and the translation of feedback into concrete plans of action.

Method: We conducted a quasi-experiment, inviting two groups of 10 clinical teachers each (1) to complete a self-evaluation and (2) subsequently examine their student ratings. One group participated in (3) an additional peer group reflection meeting. All participants were finally requested to define plans for improvement and evaluate each activity’s effectiveness.

Results: Participants perceived all three activities to be effective. Levels of reflection did not differ across the two groups. However, participation in peer group reflection did result in generating more concrete plans to change clinical teaching.

Conclusions: Peer group reflection on student ratings shows promise as tool to assist clinical teachers in generating plans for improvement. Future research should focus on whether teaching indeed improves with the introduction of peer group reflection.     

Why do surgeons teach? A qualitative analysis of motivation in excellent surgical educators

Introduction: Given that teaching is so vital to the maintenance of the medical profession, it is surprising that few authors have examined the factors which motivate physicians and surgeons to engage in this activity.

Aim: It was the aim of this study to examine the factors which motivate excellent surgical educators to teach.

Methodology: Grounded theory methodology was used to analyze transcribed semi-structured interviews. The top 20 ranked surgical educators at the University of Alberta were invited to participate. In total, 15 surgeons of various specialties were interviewed.

Results: There were five main factors which motivate surgeons to teach. These were: (1) a sense of responsibility to teach future physicians (2) an intrinsic enjoyment of teaching (3) the need to maintain and expand one’s own knowledge base (4) watching students develop into competent practicing physicians and playing a role in their success, and (5) fostering positive lifelong professional relationships with learners.

Discussion: This is the first study in surgery to use an explorative qualitative methodology. This framework of motivating factors can be used to guide professional development activities. The framework represents motivating factors for those known to be effective teachers only. We plan to use this information to engage more surgeons in teaching by providing a basis for promoting education as part of a busy surgical practice.     

Comparing a longitudinal integrated clerkship with traditional hospital-based rotations in a rural setting

Context: Longitudinal integrated clerkships (LIC) are widely used as an educational method, particularly in rural areas. They are good for facilitating hands-on learning and deep relationships between student, patients, and supervisors.

Objectives: This study sought to examine and compare learning experience of third-year rural medical students studying specialties (women’s health, aged care, child and adolescent heath, mental health, general practice) by either a traditional hospital-based rotation or a LIC in a rural general practice setting.

Methods: Data was collected from two groups of rural students (LIC; traditional hospital-based) over two academic years, utilizing focus groups to investigate general experiences of living and learning rurally, within the different educational models.

Results: Results reaffirmed that there was no perceived academic disadvantage to studying medicine rurally. Studying medicine in a rural area provides increased access to patients, more hands-on experience, and close relationships with patients and colleagues. LIC students reported increased confidence in clinical skills, felt better prepared for internship, however experienced more social isolation than students in hospital-based rotations.

Conclusions: Students undergoing a rural LIC feel more confident in their clinical skills and preparedness for practice than other rural students. This study supports the use of LICs as a powerful educational tool.