Dynamics of career choice among students in undergraduate medical courses. A BEME systematic review: BEME Guide No. 33

Introduction: Due to the lack of a theoretically embedded overview of the recent literature on medical career decision-making, this study provides an outline of these dynamics. Since differences in educational routes to the medical degree likely affect career choice dynamics, this study focuses on medical career decision-making in educational systems with a Western European curriculum structure.

Methods: A systematic search of electronic databases (Medline, Embase) was conducted from January 2008 to November 2014. A panel of seven independent reviewers performed the data extraction, quality assessment and data synthesis using the Bland–Meurer model of medical specialty choice as a reference.

Results: Fifty-seven studies met the inclusion criteria for the review. Factors associated with specialty preference or career choice can be classified in five main categories: (1) medical school characteristics (e.g., curriculum structure), (2) student characteristics (e.g., age, personality), (3) student values (e.g., personal preference), (4) career needs to be satisfied (e.g., expected income, status, and work–life balance), and (5) perception of specialty characteristics (e.g., extracurricular or curricular experiences). Especially career needs and perceptions of specialty characteristics are often associated with medical career decision-making.

Conclusion: Our results support that medical career decisions are formed by a matching of perceptions of specialty characteristics with personal needs. However, the process of medical career decision-making is not yet fully understood. Besides identifying possible predictors, future research should focus on detecting interrelations between hypothesized predictors and identify the determinants and interrelations at the various stages of the medical career decision-making process.     

A BEME systematic review of the effects of interprofessional education: BEME Guide No. 39

Background: Interprofessional education (IPE) aims to bring together different professionals to learn with, from, and about one another in order to collaborate more effectively in the delivery of safe, high-quality care for patients/clients. Given its potential for improving collaboration and care delivery, there have been repeated calls for the wider-scale implementation of IPE across education and clinical settings. Increasingly, a range of IPE initiatives are being implemented and evaluated which are adding to the growth of evidence for this form of education.

Aim: The overall aim of this review is to update a previous BEME review published in 2007. In doing so, this update sought to synthesize the evolving nature of the IPE evidence.

Methods: Medline, CINAHL, BEI, and ASSIA were searched from May 2005 to June 2014. Also, journal hand searches were undertaken. All potential abstracts and papers were screened by pairs of reviewers to determine inclusion. All included papers were assessed for methodological quality and those deemed as “high quality” were included. The presage–process–product (3P) model and a modified Kirkpatrick model were employed to analyze and synthesize the included studies.

Results: Twenty-five new IPE studies were included in this update. These studies were added to the 21 studies from the previous review to form a complete data set of 46 high-quality IPE studies. In relation to the 3P model, overall the updated review found that most of the presage and process factors identified from the previous review were further supported in the newer studies. In regard to the products (outcomes) reported, the results from this review continue to show far more positive than neutral or mixed outcomes reported in the included studies. Based on the modified Kirkpatrick model, the included studies suggest that learners respond well to IPE, their attitudes and perceptions of one another improve, and they report increases in collaborative knowledge and skills. There is more limited, but growing, evidence related to changes in behavior, organizational practice, and benefits to patients/clients.

Conclusions: This updated review found that key context (presage) and process factors reported in the previous review continue to have resonance on the delivery of IPE. In addition, the newer studies have provided further evidence for the effects on IPE related to a number of different outcomes. Based on these conclusions, a series of key implications for the development of IPE are offered.     

The failure to fail underperforming trainees in health professions education: A BEME systematic review: BEME Guide No. 42

Background: Many clinical educators feel unprepared and/or unwilling to report unsatisfactory trainee performance. This systematic review consolidates knowledge from medical, nursing, and dental literature on the experiences and perceptions of evaluators or assessors with this failure to fail phenomenon.

Methods: We searched the English language literature in CINAHL, EMBASE, and MEDLINE from January 2005 to January 2015. Qualitative and quantitative studies were included. Following our review protocol, registered with BEME, reviewers worked in pairs to identify relevant articles. The investigators participated in thematic analysis of the qualitative data reported in these studies. Through several cycles of analysis, discussion and reflection, the team identified the barriers and enablers to failing a trainee.

Results: From 5330 articles, we included 28 publications in the review. The barriers identified were (1) assessor’s professional considerations, (2) assessor’s personal considerations, (3) trainee related considerations, (4) unsatisfactory evaluator development and evaluation tools, (5) institutional culture and (6) consideration of available remediation for the trainee. The enablers identified were: (1) duty to patients, to society, and to the profession, (2) institutional support such as backing a failing evaluation, support from colleagues, evaluator development, and strong assessment systems, and (3) opportunities for students after failing.

Discussion/conclusions: The inhibiting and enabling factors to failing an underperforming trainee were common across the professions included in this study, across the 10 years of data, and across the educational continuum. We suggest that these results can inform efforts aimed at addressing the failure to fail problem.     

Evidence regarding the utility of multiple mini-interview (MMI) for selection to undergraduate health programs: A BEME systematic review: BEME Guide No. 37

Background: In the 11 years since its development at McMaster University Medical School, the multiple mini-interview (MMI) has become a popular selection tool. We aimed to systematically explore, analyze and synthesize the evidence regarding MMIs for selection to undergraduate health programs.

Methods: The review protocol was peer-reviewed and prospectively registered with the Best Evidence Medical Education (BEME) collaboration. Thirteen databases were searched through 34 terms and their Boolean combinations. Seven key journals were hand-searched since 2004. The reference sections of all included studies were screened. Studies meeting the inclusion criteria were coded independently by two reviewers using a modified BEME coding sheet. Extracted data were synthesized through narrative synthesis.

Results: A total of 4338 citations were identified and screened, resulting in 41 papers that met inclusion criteria. Thirty-two studies report data for selection to medicine, six for dentistry, three for veterinary medicine, one for pharmacy, one for nursing, one for rehabilitation, and one for health science. Five studies investigated selection to more than one profession. MMIs used for selection to undergraduate health programs appear to have reasonable feasibility, acceptability, validity, and reliability. Reliability is optimized by including 7–12 stations, each with one examiner. The evidence is stronger for face validity, with more research needed to explore content validity and predictive validity. In published studies, MMIs do not appear biased against applicants on the basis of age, gender, or socio-economic status. However, applicants of certain ethnic and social backgrounds did less well in a very small number of published studies. Performance on MMIs does not correlate strongly with other measures of noncognitive attributes, such as personality inventories and measures of emotional intelligence.

Discussion: MMI does not automatically mean a more reliable selection process but it can do, if carefully designed. Effective MMIs require careful identification of the noncognitive attributes sought by the program and institution. Attention needs to be given to the number of stations, the blueprint and examiner training.

Conclusion: More work is required on MMIs as they may disadvantage groups of certain ethnic or social backgrounds. There is a compelling argument for multi-institutional studies to investigate areas such as the relationship of MMI content to curriculum domains, graduate outcomes, and social missions; relationships of applicants’ performance on different MMIs; bias in selecting applicants of minority groups; and the long-term outcomes appropriate for studies of predictive validity.     

Test-enhanced learning in health professions education: A systematic review: BEME Guide No. 48

Background: Cognitive psychology studies demonstrate that subjects who attempt to recall information show better learning, retention, and transfer than subjects who spend the same time studying the same material (test-enhanced learning, TEL). We systematically reviewed TEL interventions in health professions education.

Methods: We searched 13 databases, 14 medical education journals, and reference lists. Inclusion criteria included controlled studies of TEL that compared TEL to studying the same material or to a different TEL strategy. Two raters screened articles for inclusion, abstracted information, determined quality scores, and calculated the standardized mean difference (SMD) for the learning outcomes.

Results: Inter-rater agreement was excellent for all comparisons. The 19 included studies reported 41 outcomes with data sufficient to determine a SMD. TEL interventions included short answer questions, multiple choice questions, simulation, and standardized patients. Five of six immediate learning outcomes (SMD 0.09–0.44), 21 of 23 retention outcomes (SMD 0.12–2.5), and all seven transfer outcomes (SMD 0.33–1.1) favored TEL over studying.

Conclusions: TEL demonstrates robust effects across health professions, learners, TEL formats, and learning outcomes. The effectiveness of TEL extends beyond knowledge assessed by examinations to clinical applications. Educators should include TEL in health professions curricula to enhance recall, retention, and transfer.     

A best evidence systematic review of interprofessional education: BEME Guide no. 9

BACKGROUND AND REVIEW CONTEXT: Evidence to support the proposition that learning together will help practitioners and agencies work better together remains limited and thinly spread. This review identified, collated, analysed and synthesised the best available contemporary evidence from 21 of the strongest evaluations of IPE to inform the above proposition. In this way we sought to help shape future interprofessional education and maximize the potential for interprofessional learning to contribute to collaborative practice and better care. OBJECTIVES OF THE REVIEW: To identify and review the strongest evaluations of IPE. To classify the outcomes of IPE and note the influence of context on particular outcomes. To develop a narrative about the mechanisms that underpin and inform positive and negative outcomes of IPE. SEARCH STRATEGY: Bibliographic database searches as follows: Medline 1966-2003, CINAHL 1982-2001, BEI 1964-2001, ASSIA 1990-2003 which produced 10,495 abstracts. Subsequently, 884 full papers were obtained and scrutinized. In addition, hand searching (2003-5 issues) of 21 journals known to have published two or more higher quality studies from a previous review. TOPIC DEFINITION AND INCLUSION CRITERIA: Peer-reviewed papers and reports included in the review had to be formal educational initiatives attended by at least two of the many professional groups from health and social care, with the objective of improving care; and learning with, from and about each other. DATA COLLECTION, ANALYSIS AND SYNTHESIS: Standard systematic review procedures were applied for sifting abstracts, scrutinizing full papers and abstracting data. Two members of the team checked each abstract to decide whether the full paper should be read. A third member was consulted over any discrepancies. Similarly, each full paper was read by at least two members of the team and agreement sought before passing it to one member of the team (SR) for data abstraction. Other members of the team checked 10% of the abstraction records. Coding into a Statistical Package for Social Scientists (SPSS) data base led to collection of different outcome measures used in the primary studies via the common metric of an adapted Kirkpatrick's four-level model of educational outcomes. Additionally, a narrative synthesis was built after analysis of primary data with the 3-P model (presage-process-product) of education development and delivery. HEADLINE RESULTS: Government calls for enhanced collaboration amongst practitioners frequently leads to IPE that is then developed and delivered by educators, practitioners or service managers. Staff development is a key influence on the effectiveness of IPE for learners who all have unique values about themselves and others. Authenticity and customization of IPE are important mechanisms for positive outcomes of IPE. Interprofessional education is generally well received, enabling knowledge and skills necessary for collaborative working to be learnt; it is less able to positively influence attitudes and perceptions towards others in the service delivery team. In the context of quality improvement initiatives interprofessional education is frequently used as a mechanism to enhance the development of practice and improvement of services.     

Learning from patients about patient-centredness: A realist review: BEME Guide No. 60

Abstract

Background: Patient-centred work is an essential part of contemporary medicine. Literature shows that educational interventions contribute to developing patient-centredness, but there is a lack of insight into the associated learning processes.

Objective: Through reviewing articles about educational interventions involving patients, we aspire to develop a program theory that describes the processes through which the educational interventions are expected to result in change. The processes will clarify contextual elements (called contexts) and mechanisms connected to learning patient-centredness.

Methods: In our realist review, an initial, rough program theory was generated during the scoping phase, we searched for relevant articles in PubMed, PsycINFO, ERIC, CINAHL and Embase for all years before and through 2016. We included observational studies, case reports, interviews, and experimental studies in which the participants were students, residents, doctors, nurses or dentists. The relevance and rigour of the studies were taken into account during analysis. With deductive as well as inductive coding, we extended the rough program theory.

Results: In our review, we classified five different contexts which affect how upcoming professionals learn patient-centredness. These aspects are influenced through components in the intervention(s) related to the learner, the teacher, and the patient. We placed the mechanisms together in four clusters – comparing and combining as well as broadening perspectives, developing narratives and engagement with patients, self-actualisation, and socialisation – to show how the development of (dimensions of) patient-centredness occurs. Three partial-program-theories (that together constituting a whole program theory) were developed, which show how different components of interventions within certain contexts will evoke mechanisms that contribute to patient-centredness.

Translation into daily practice: These theories may help us better understand how the roles of patients, learners and teachers interact with contexts such as the kind of knowledge that is considered legitimate or insight in the whole illness trajectory. Our partial program theories open up potential areas for future research and interventions that may benefit learners, teachers, and patients.     

Tools for structured team communication in pre-registration health professions education: a Best Evidence Medical Education (BEME) review: BEME Guide No. 41

Introduction: Calls for the inclusion of standardized protocols for information exchange into pre-registration health professions curricula have accompanied their introduction into clinical practice. In order to help clinical educators respond to these calls, we have reviewed educational interventions for pre-registration students that incorporate one or more of these ?tools for structured communication?.

Methods: Searches of 10 databases (1990?2014) were supplemented by hand searches and by citation searches (to January 2015). Studies evaluating an intervention for pre-registration students of any clinical profession and incorporating at least one tool were included. Quality of included studies was assessed using a checklist of 11 indicators and a narrative synthesis of findings undertaken.

Results: Fifty studies met our inclusion criteria. Of these, 21 evaluated the specific effect of a tool on educational outcomes, and 27 met seven or more quality indicators.

Conclusions: Pre-registration students, particularly those in the US, are learning to use tools for structured communication either in specific sessions or integrated into more extensive courses or programmes; mostly ‘Situation Background Assessment Recommendation’ and its variants. There is some evidence that learning to use a tool can improve the clarity and comprehensiveness of student communication, their perceived self-confidence and their sense of preparedness for clinical practice. There is, as yet, little evidence for the transfer of these skills to the clinical setting or for any influence of teaching approach on learning outcomes. Educators will need to consider the positioning of such learning with other skills such as clinical reasoning and decision-making.     

Cognitive load theory for training health professionals in the workplace: A BEME review of studies among diverse professions: BEME Guide No. 53

Aim: Cognitive load theory (CLT) is of increasing interest to health professions education researchers. CLT has intuitive applicability to workplace settings, yet how CLT should inform teaching, learning, and research in health professions workplaces is unclear.

Method: To map the existing literature, we performed a scoping review of studies involving cognitive load, mental effort and/or mental workload in professional workplace settings within and outside of the health professions. We included actual and simulated workplaces and workplace tasks.

Result: Searching eight databases, we identified 4571 citations, of which 116 met inclusion criteria. Studies were most often quantitative. Methods to measure cognitive load included psychometric, physiologic, and secondary task approaches. Few covariates of cognitive load or performance were studied. Overall cognitive load and intrinsic load were consistently negatively associated with the level of experience and performance. Studies consistently found distractions and other aspects of workplace environments as contributing to extraneous load. Studies outside the health professions documented similar findings to those within the health professions, supporting relevance of CLT to workplace learning.

Conclusion: The authors discuss implications for workplace teaching, curricular design, learning environment, and metacognition. To advance workplace learning, the authors suggest future CLT research should address higher-level questions and integrate other learning frameworks.     

Utility of selection methods for specialist medical training: A BEME (best evidence medical education) systematic review: BEME guide no. 45

Background: Selection into specialty training is a high-stakes and resource-intensive process. While substantial literature exists on selection into medical schools, and there are individual studies in postgraduate settings, there seems to be paucity of evidence concerning selection systems and the utility of selection tools in postgraduate training environments.

Aim: To explore, analyze and synthesize the evidence related to selection into postgraduate medical specialty training.

Method: Core bibliographic databases including PubMed; Ovid Medline; Embase, CINAHL; ERIC and PsycINFO were searched, and a total of 2640 abstracts were retrieved. After removing duplicates and screening against the inclusion criteria, 202 full papers were coded, of which 116 were included.

Results: Gaps in underlying selection frameworks were illuminated. Frameworks defined by locally derived selection criteria, and heavily weighed on academic parameters seem to be giving way to the evidencing of competency-based selection approaches in some settings.

Regarding selection tools, we found favorable psychometric evidence for multiple mini-interviews, situational judgment tests and clinical problem-solving tests, although the bulk of evidence was mostly limited to the United Kingdom. The evidence around the robustness of curriculum vitae, letters of recommendation and personal statements was equivocal. The findings on the predictors of past performance were limited to academic criteria with paucity of long-term evaluations. The evidence around nonacademic criteria was inadequate to make an informed judgment.

Conclusions: While much has been gained in understanding the utility of individual selection methods, though the evidence around many of them is equivocal, the underlying theoretical and conceptual frameworks for designing holistic and equitable selection systems are yet to be developed.     

BEME Guide No. 1: Best Evidence Medical Education

There is a need to move from opinion-based education to evidence-based education. Best evidence medical education (BEME) is the implementation, by teachers in their practice, of methods and approaches to education based on the best evidence available. It involves a professional judgement by the teacher about his/her teaching taking into account a number of factors-the QUESTS dimensions. The Quality of the research evidence available-how reliable is the evidence? the Utility of the evidence-can the methods be transferred and adopted without modification, the Extent of the evidence, the Strength of the evidence, the Target or outcomes measured-how valid is the evidence? and the Setting or context-how relevant is the evidence? The evidence available can be graded on each of the six dimensions. In the ideal situation the evidence is high on all six dimensions, but this is rarely found. Usually the evidence may be good in some respects, but poor in others.The teacher has to balance the different dimensions and come to a decision on a course of action based on his or her professional judgement.The QUESTS dimensions highlight a number of tensions with regard to the evidence in medical education: quality vs. relevance; quality vs. validity; and utility vs. the setting or context. The different dimensions reflect the nature of research and innovation. Best Evidence Medical Education encourages a culture or ethos in which decision making takes place in this context.     

Workplace learning through collaboration in primary healthcare: A BEME realist review of what works,for whom and in what circumstances: BEME Guide No. 46

Background: Changes in healthcare practice toward more proactive clinical, organizational and interprofessional working require primary healthcare professionals to learn continuously from each other through collaboration. This systematic review uses realist methodology to consolidate knowledge on the characteristics of workplace learning (WPL) through collaboration by primary healthcare professionals.

Methods: Following several scoping searches, five electronic bibliographic databases were searched from January 1990 to December 2015 for relevant gray and published literature written in English, French, German and Dutch. Reviewers worked in pairs to identify relevant articles. A set of statements, based on the findings of our scoping searches, was used as a coding tree to analyze the papers. Interpretation of the results was done in alternating pairs, discussed within the author group and triangulated with stakeholders’ views.

Results: Out of 6930 references, we included 42 publications that elucidated who, when, how and what primary healthcare professionals learn through collaboration. Papers were both qualitative and quantitative in design, and focused largely on WPL of collaborating general practitioners and nurses. No striking differences between different professionals within primary healthcare were noted. Professionals were often unaware of the learning that occurs through collaboration. WPL happened predominantly through informal discussions about patient cases and modeling for other professionals. Any professionals could both learn and facilitate others’ learning. Outcomes were diverse, but contextualized knowledge seemed to be important.

Discussion/conclusions: Primary care professionals’ WPL is multifaceted. Existing social constructivist and social cognitivist learning theories form a framework from which to interpret these findings. Primary care policy makers and managers should ensure that professionals have access to protected time, earmarked for learning. Time is required for reflection, to learn new ways of interaction and to develop new habits within clinical practice.