Faculty training in evidence-based medicine: improving evidence acquisition and critical appraisal

INTRODUCTION: Evidence-based medicine (EBM) integrates published clinical evidence with patient values and clinical expertise, the output of which is informed medical decision making. Key skills for evidence-based practice include acquisition and appraisal of clinical information. Faculty clinicians often lack expertise in these skills and are therefore unable to demonstrate this process for students and residents. METHODS: We conducted a yearlong case-based EBM workshop for 28 clinician educators, with precourse and postcourse evaluations of EBM resource use and literature appraisal skills. RESULTS: Of the original 28 participants, 26 completed the course. Self-assessed EBM resource use improved significantly. Self-reported EBM knowledge correlated with measured skill (r = 0.45), and both improved with the intervention (both p < .001). Higher EBM skills scores correlated with time logged on the course's EBM Web sites (r = 0.56; p < .05), workshop attendance rates (r = 0.55; p = .003), and fewer years since medical school graduation (r = -0.56; p < .005). DISCUSSION: An interactive, longitudinal, EBM course derived from a needs assessment can improve 2 skills important for evidence-based practice: online literature retrieval and critical appraisal skills.     

Teaching evidence-based medicine more effectively

Evidence-based Medicine (EBM) is becoming an integral component of graduate medical education competency and a requirement for grad medical education practice-based learning core competency. This study tries to compare the efficacy of conferences utilizing small-group discussions with the traditional conference method in enhancing EBM competency. The participants in this randomized controlled trial (RCT) were 170 members of the medical faculty who were divided into two groups of 86 (intervention) and 84 (control). Following the intervention, EBM competency was assessed by a written examination. statistical analysis made use of chi-square test, independent sample t-test and relative risks for univariate analysis. Mantel-Hanszel was used for bivariate analysis. Cox proportional hazard models were used to evaluate multivariate-adjusted associations between EBM educational intervention and EBM knowledge, attitude and skills. A new indicator of number needed to intervention (NNI) was defined and computed. Results: The results proved conference along with small-group discussion to be a more effective teaching method with P=0.001 on knowledge, P<0.001 for attitude and skills P<0.001 in an EBM exam when compared with medical faculty members who did not participate in EBM educational intervention (n=84). Moreover, they had also increased confidence with critical appraisal skills, and searching EBM resources. Conclusions: Conferences followed by small-group discussions significantly enhance EBM knowledge, attitude, critical appraisal skills and literature review skills.     

Comparison of directed and self-directed learning in evidence-based medicine: a randomised controlled trial

OBJECTIVES: To compare 2 educational programmes for teaching evidence-based medicine (EBM). DESIGN: Prospective randomised controlled trial accompanied by a qualitative evaluation. SETTING: University of Oslo, Norway, 2002-03. PARTICIPANTS: A total of 175 students entered the study. All tenth semester medical students from 3 semesters were eligible for inclusion if they completed baseline assessment and consent forms and either attended teaching on the first day of the semester or gave reasons for their absence on the first day in advance. Interventions One intervention was based on computer-assisted, self-directed learning (self-directed intervention), whilst the other was organised as workshops based on social learning theory (directed intervention). Both educational interventions consisted of 5 half-day sessions. MAIN OUTCOME MEASURES: The primary outcomes were knowledge about EBM and skills in critical appraisal. A secondary outcome measured attitudes to EBM. Outcomes were compared on an intention-to-treat basis using a stratified Wilcoxon rank-sum test. RESULTS: There were no differences in outcomes for the 2 study groups in terms of EBM knowledge (mean deviation 0.0 [95% confidence interval - 1.0, 1.0], P = 0.8), critical appraisal skills (MD 0.1 [95% CI - 0.9, 1.1], P = 0.5), or attitudes to EBM (MD - 0.3 [95% CI - 1.4, 0.8], P = 0.5). Follow-up rates were 96%, 97% and 63%, respectively. CONCLUSIONS: This trial and its accompanying qualitative evaluation suggest that self-directed, computer-assisted learning may be an alternative format for teaching EBM. However, further research is needed to confirm this and investigate alternative educational models.     

Preliminary evaluation of "interpreter" role plays in teaching communication skills to medical undergraduates

RATIONALE AND OBJECTIVES: Multiculturalism presents linguistic obstacles to health care provision. We explored the early introduction of "interpreter" role-play exercises in teaching medical undergraduates communication skills. The interpreter role creates a natural barrier in communication providing an active prompt for recognizing learning needs in this area. METHODS: Bilingual Cantonese first-year medical students (n=160) were randomly allocated to either "Observer" or "Interpreter" role plays at a small-group introductory communication skills workshop using a quasi experimental design, counterbalanced across tutors. Students assessed their own skill competence before and, together with their perceptions of the different role plays' effectiveness, again after the workshop, using an anonymous 16 item Likert-type scale, analysed using ANOVA and MANOVA. RESULTS: Students' assessments of their skills improved significantly following the workshop (F=73.19 [1,156], P=0.0009). Students in the observer group reported greater changes in their scores following the workshop than did students in the interpreter group (F=4.84 [1,156], P=0.029), largely due to improvement in perceived skill (F=4.38 [1,156], P=0.038) rather than perceived programme effectiveness (F=3.13 [1,156], P > 0.05). Subsequent MANOVA indicated no main effect of observer/interpreter conditions, indicating these differences could be attributed to chance alone (F=1.41 [16 141], P > 0.05). CONCLUSION: The workshop positively influenced students' perceived communication skills, but the "Interpreter" role was less effective than the "Observer" role in achieving this. Future studies should examine whether interpreter role plays introduced later in the medical programme are beneficial.     

Teaching patient wellness to first-year medical students: the impact on future ability to perform the history of present illness

OBJECTIVES: Curriculum innovations to improve clinical skills have been implemented at many American medical schools. A current curricular change at the University of Connecticut School of Medicine involves teaching wellness to students in the first year rather than the more traditional focus on disease processes. It is unknown, however, if focusing on wellness detracts from students' future ability to perform the history of present illness (HPI) which requires students to focus on disease processes. DESIGN: The current study examined this issue by comparing two cohorts' clinical skills (n=156), with one class participating in a traditional curriculum during their first year and the other in a revised curriculum teaching wellness during the first year. Each class was evaluated at the beginning of their second year to determine their level of clinical competence. SETTING: University of Connecticut School of Medicine SUBJECTS: Second-year medical students. RESULTS: Analyses suggested that teaching wellness did not detract from future ability to perform an HPI, and in fact students taught wellness had significantly higher history-taking scores. CONCLUSIONS: Curricular innovations which stress wellness and prevention early in medical education do not detract from and may enhance students' ability to perform the history of present illness later during medical training.     

Use of a journal club and letter-writing exercise to teach critical appraisal to medical undergraduates

INTRODUCTION: There is growing interest in methods of teaching critical appraisal skills at undergraduate and postgraduate levels. We describe an approach using a journal club and subsequent letter writing to teach critical appraisal and writing skills to medical undergraduates. METHODS: The exercise occurs during a 3-week public health medicine attachment in the third year of the undergraduate curriculum. Students work in small groups to appraise a recently published research paper, present their findings to their peers in a journal club, and draft a letter to the journal editor. Evaluation took place through: informal and formal feedback from students; number of letters written, submitted and published, and a comparison of marks obtained by students submitting a literature review assignment with and without critical appraisal teaching during the public health attachment. RESULTS: Feedback from students was overwhelmingly positive. In the first 3(1/2) years, 26 letters have been published or accepted for publication, and 58 letters published on the Internet. There were no significant differences in overall marks or marks for the critical appraisal component of the literature review assignments between the two student groups. DISCUSSION: We believe our approach is an innovative and enjoyable method for teaching critical appraisal and writing skills to medical students. Lack of difference in marks in the literature review between the student groups may reflect its insensitivity as an outcome measure, contamination by other critical appraisal teaching, or true ineffectiveness.     

Change in knowledge and skills of Year 3 undergraduates in evidence-based medicine seminars

PURPOSE: To evaluate the effect of a compulsory evidence-based medicine (EBM) seminar in critical appraisal skills and the overall acceptance of compulsory EBM seminars for Year 3 medical undergraduate students. METHODS: Small group seminars by peer teaching were conducted for up to 23 undergraduates. Knowledge and skills in EBM before and after the compulsory seminars were evaluated by 2 different sets of 20 questions. To apply knowledge, each undergraduate had to analyse an individual paper case using the principles of EBM. Undergraduates gave anonymous feedback using separate evaluation sheets at the end of the seminar. Main outcome variables were changes in knowledge and skills. RESULTS: A total of 132 Year 3 undergraduates at the University of Frankfurt participated in a compulsory EBM seminar during the academic half-year 2003/04 as part of their regular curriculum. Complete datasets were available for evaluation from 124 undergraduates (94%). The seminars led to an overall increase in knowledge (question paper score increase from 2.37 to 7.48, 99% CI 6.61-8.36, or 216%). Transfer of knowledge into a paper case scenario was generally good, with a mean score of 49.5 (SD 5.24) out of 55 points. Feedback indicated good overall acceptance of the seminars, with a median of 2 (score range from 1 = excellent to 6 = failed). CONCLUSION: Trained medical students are effective and well accepted EBM trainers in compulsory undergraduate seminars.     

Variation in medical students'' approaches to diagnosis: a basis for initiating conceptual change among teachers and students

OBJECTIVES: This study examines Finnish medical students' approaches to diagnosis and investigates further how medical teachers can use information about variation in their students' approaches to diagnosis to foster teaching in medical school. DESIGN: The medical students responded to the Conceptions and Experiences of Diagnosis Inventory (CEDI). SETTING: Faculty of Medicine, University of Helsinki. SUBJECTS: Ninety medical students in their clinical years and eight clinical teachers from the same Faculty of Medicine. RESULTS: The 11 subscales of the CEDI formed two contrasting factors: the first reflecting variation in 'non-virtuously' labelled, and the second in 'virtuously' labelled, aspects of diagnosis. CONCLUSIONS: Cluster analyses revealed subgroup characteristics of students' diagnostic processes that are of potential benefit to both students and teachers. Teacher interviews indicated that, for students, the CEDI may act as a self-assessment tool to help develop their diagnostic and metacognitive skills. For teachers, the CEDI was seen to offer important information about their students' conceptions of diagnosis and diagnostic skills.     

Teaching and evaluating first and second year medical students' practice of evidence-based medicine

PURPOSE: To implement an evidence-based medicine (EBM) curriculum for Year 1 and 2 medical students, and to develop a method to evaluate their practice of EBM in discrete and relevant worksteps. METHODS: For the 100 students entering Year 1 of their medical education in 2000, we implemented a curriculum with 25-30 student contact hours of EBM instruction which used a variety of teaching formats and spanned the first and second years of their training. We developed an evaluation module that assessed the following 5 steps in the practice of EBM: generating well built questions; searching for evidence; critical appraisal; applying the evidence, and self-evaluation. We tested 2 different versions of the test module 3-months apart with the same cohort of second year students, and correlated their scores on the second module with examination components of a comprehensive assessment. We obtained feedback from the students regarding the EBM curriculum and evaluation method. RESULTS: Each test module took 2-4 hours to complete and 5-8 minutes to grade. There was moderate test-retest reliability for the total test scores (r = 0.35, P < 0.001). Step 1 scores correlated with the mock board examination scores (r = 0.23, P = 0.05). Step 2 scores correlated with the peer assessment factor "work habits" (r = 0.24, P = 0.02), and Step 3 scores correlated with clinical reasoning exercises (r = 0.31, P = 0.002). Step 4 scores lacked test-retest reliability and did not correlate with components of the comprehensive assessment. The majority of students felt there was too much focus on EBM during the first 2 years of the curriculum and they rated the EBM test module the lowest rated component of the comprehensive assessment. CONCLUSIONS: Although we have demonstrated preliminary reliability and validity of a new evaluation instrument that assess the domains of scientific knowledge, work habits and reasoning skills required in the practice of EBM, many of the correlations were weak, and we remain in the very early stages of determining if, when and how EBM instruction should occur in medical education.     

Assessing the development of communication skills in undergraduate medical students

CONTEXT: The teaching of clinical communication skills' teaching has become an important part of medical school curricula. Many undergraduate medical courses include communication skills training at various points in their curriculum. Very few reports have been published on the development of communication skills over the duration of a medical undergraduate training. AIMS: To determine the change in communication skills between early and mid-stages of the students' 5-year curriculum, and to investigate the predictive and theoretical significance of knowledge and understanding of communication skills in relation to observed performance. PARTICIPANTS: Students entering as the first cohort to the new medical curriculum at Liverpool Medical School (n=207). Nine students withdrew leaving 198 students who completed two summative assessments in June 1997 (level 1) and November 1998 (level 2). STATISTICAL ANALYSIS: Repeated measures multivariate ANOVAS were applied to the main study data to detect any change in performance between levels 1 and 2. RESULTS AND CONCLUSIONS: An improvement in communication skills was found in medical students over 17 months of their undergraduate teaching: that is from the level 1 to the level 2 assessment. Knowledge and understanding of communication skills at initial assessment did not show the predicted association with performance at level 2.     

Integration of evidence based medicine into the clinical years of a medical curriculum

Teaching Evidence Based Medicine (EBM) helps medical students to develop their decision making skills based on current best evidence, especially when it is taught in a clinical context. Few medical schools integrate Evidence Based Medicine into undergraduate curriculum, and those who do so, do it at the academic years only as a standalone (classroom) teaching but not at the clinical years. The College of Medicine at King Saud bin Abdulaziz University for Health Sciences was established in January 2004. The college adopted a four-year Problem Based Learning web-based curriculum. The objective of this paper is to present our experience in the integration of the EBM in the clinical phase of the medical curriculum. We teach EBM in 3 steps: first step is teaching EBM concepts and principles, second is teaching the appraisal and search skills, and the last step is teaching it in clinical rotations. Teaching EBM at clinical years consists of 4 student-centered tutorials. In conclusion, EBM may be taught in a systematic, patient centered approach at clinical rounds. This paper could serve as a model of Evidence Based Medicine integration into the clinical phase of a medical curriculum.     

Evaluation of procedural skills training in an undergraduate curriculum

CONTEXT: A substantial proportion of medical students enter their intern year without any basic skills experience. Lack of experience is a significant source of stress for many junior doctors. OBJECTIVES: To evaluate the effect of a basic procedural skills tutorial for Year 3 medical students on their competence in relevant skills at Year 5. SUBJECTS: The control group consisted of 93 medical students who completed Year 3 in 1996. The intervention group consisted of 92 medical students who completed Year 3 in 1997. The intervention group received a practical skills tutorial in Year 3; the control group did not. Both groups were assessed on their practical skills competence during Year 5. METHODS: A 3-hour practical tutorial on injection and suturing techniques was delivered to the intervention group. The effectiveness of the intervention was assessed by self-reported experience of giving injections, inserting sutures and sustaining needlestick injuries, and by teacher-rated competency in four basic procedural skills. RESULTS: Students who received the Year 3 tutorial were significantly more likely to record a satisfactory assessment for their performance in all four basic skills compared with students who did not receive the tutorial. They were less likely than controls to refuse invitations to give injections, but not invitations to insert a suture, during Years 4 and 5. CONCLUSIONS: A single session of formalised teaching in procedural skills in the early stages of a medical degree can have long-term effectiveness in basic skills competence and may increase students' confidence to practise their skills.     

Learning to consult with computers

OBJECTIVE: To develop and evaluate a strategy to teach skills and issues associated with computers in the consultation. INTERVENTION: An overview lecture plus a workshop before and a workshop after practice placements, during the 10-week general practice (GP) term in the 5th year of the University of Melbourne medical course. DESIGN: Pre- and post-intervention study using a mix of qualitative and quantitative methods within a strategic evaluation framework. OUTCOME MEASURES: Self-reported attitudes and skills with clinical applications before, during and after the intervention. RESULTS: Most students had significant general computer experience but little in the medical area. They found the workshops relevant, interesting and easy to follow. The role-play approach facilitated students' learning of relevant communication and consulting skills and an appreciation of issues associated with using the information technology tools in simulated clinical situations to augment and complement their consulting skills. The workshops and exposure to GP systems were associated with an increase in the use of clinical software, more realistic expectations of existing clinical and medical record software and an understanding of the barriers to the use of computers in the consultation. CONCLUSIONS: The educational intervention assisted students to develop and express an understanding of the importance of consulting and communication skills in teaching and learning about medical informatics tools, hardware and software design, workplace issues and the impact of clinical computer systems on the consultation and patient care.     

Early introduction of clinical skills teaching in a medical curriculum--factors affecting students' learning

OBJECTIVE: To evaluate the effects of the early introduction of clinical skills teaching on students' learning following an overhaul of the curriculum of a traditional Asian medical school. METHODS: Randomly selected medical students in Year I and II were invited to participate in 30 focus group interviews while all students were asked to assist with the questionnaire survey. Most students were contacted personally to help them understand the objectives of the study. Confidentiality was emphasised and a non-faculty interviewer was recruited for the interviews. RESULTS: Two hundred and eight of Year I/Year II students attended the lunchtime focus group interviews (response rate=86.7%) while 252 (73.5%) students returned the questionnaire. The majority of them (87%) agreed or strongly agreed that it was good to introduce clinical skills in the early years of the curriculum. They reflected that the course enhanced their learning interest and made them feel like doctors. They also made many constructive suggestions on how the course could be improved during the interactive focus group interviews so that the negative effects could be minimised. CONCLUSION: It is useful to introduce clinical skills in the early years of a medical curriculum. A comprehensive course evaluation, using both quantitative and qualitative methods, helps to collect useful information on how the course can be improved.     

A process evaluation of medical ethics education in the first year of a new medical curriculum

BACKGROUND: Despite the recent increase in activity in the field of medical ethics education, few evaluative studies have been carried out. Most studies have taken place in North America, in curricula where teaching is discipline-based, and have concentrated on outcome rather than on the curricular processes adopted. AIM: To evaluate the process of medical ethics education in the first year of a new learner-centred, problem-based, integrated medical curriculum. METHOD: A qualitative, multi-method approach was adopted using open questionnaires, focus groups and tutor evaluation rating scales. The study involved all 238 students in the first year of the new medical curriculum, and the 30 clinical tutors who facilitated ethics learning. A stratified sampling technique was used to choose focus group participants. RESULTS: Small group teaching proved highly acceptable to both students and tutors. Tutors' teaching skills were central to its effectiveness. Tutors played an important role in promoting students' appreciation of the relevance of medical ethics to clinical practice, and in establishing a climate where constructive criticism of colleagues' actions is acceptable. Course integration, including the provision for students of clinical experiences on which to reflect, was an important aid to learning. Students and tutors were noted to be driving the ethics curriculum towards having a contextual rather than theoretical base. CONCLUSION: This evaluation identified those aspects of the medical ethics course which contributed to its effectiveness and those which detracted from it. This information will be used to inform future development.     

Teaching clinical reasoning – a preliminary controlled study

BACKGROUND: The formal and explicit teaching of clinical reasoning is rarely undertaken in medical schools despite widespread recognition that knowledge acquisition alone may be insufficient to develop good clinical judgement. Previous work studying the effects of such teaching is inconclusive. DESIGN AND METHODS: A controlled observational study of 4th year medical students considered the effect of a brief teaching intervention on clinical reasoning skills. A validated measure of clinical reasoning - the diagnostic thinking inventory - was used as the outcome measure. RESULTS: Students participating in the teaching intervention performed significantly better on the diagnostic thinking inventory than control students. CONCLUSION: Further research is necessary to evaluate the long-term impact of such teaching.     

Translating learning principles into practice: a new strategy for learning clinical skills

AIM: There are data to suggest that medical school may not adequately prepare doctors for practice and that there are deficiencies in undergraduate teaching of skills in history taking, physical examination, diagnosis and management (clinical skills). There is a need to re-evaluate methods by which we can teach clinical skills effectively. This aim of this review was to describe the literature concerning the important principles underpinning effective clinical learning. Subsequently a structured learning tool and teaching process was developed in order to support these principles. METHOD: The principles of effective clinical learning were derived after a search of the medical education and relevant behavioural science literature. Consequently, a structured learning tool and teaching process was developed in order to potentiate the translation of these principles into practice for medical school training in clinical skills. RESULTS: Ten principles were derived from the 68 articles referred to in this review. These were: making active decisions, an individual focus to learning, gaining experience, feedback to the learner, reciprocal learning, holistic care, relevant learning, feasibility, cost efficiency and mentoring. A process for history taking, physical examination and management plan was developed for medical students which incorporated these principles. CONCLUSION: Relevant literature can provide the foundations for teaching and learning methods in medical education. We plan to trial this method and evaluate the impact on student learning outcomes.     

The art and science of clinical teaching

A review of the literature on clinical teaching indicates that medical schools emphasize didactic teaching of facts. Interpersonal skills, problem solving skills and cultivation of attitudes are neglected. However, the literature also described the skills of 'good' clinical teachers, and enumerates them. Programmes attempting to develop students' interpersonal skills and problem solving skills are reported. Finally, areas for further practice and study in clinical teaching are recommended.