Should we teach using schemas? Evidence from a randomised trial

Medical Education 2012: 46: 815–822 Context Schema‐based instruction may alter knowledge organisation and diagnostic reasoning strategies through the provision of structured knowledge to novice trainees. The effects of schema‐based instruction on diagnostic accuracy and knowledge organisation have not been rigorously tested. Methods Year 2 medical students were randomised to learn four cardiac diagnoses using schema‐based instruction (n = 26) or traditional instruction (n = 27) on a high‐fidelity cardiopulmonary simulator (CPS). Students completed case‐based learning in groups of two to five and underwent individual written and practical tests. The written test consisted of questions testing features that linked or distinguished diagnoses (structured knowledge) and questions testing features of individual diagnoses (factual knowledge). A practical test of diagnostic accuracy on the CPS was performed for two diagnoses present in the learning phase (taught lesions) and two untaught lesions. A majority of students (n = 37, 70%) voluntarily returned for follow‐up written testing 2–4 weeks later. Results Learning time and accuracy did not differ between students on schema‐based and those on traditional instruction. Students receiving schema‐based instruction performed better on structured knowledge questions (p < 0.001) and no differently on factual knowledge questions (p = 0.7). Relative differences between groups remained unchanged on follow‐up testing. Diagnostic success was higher in the schema‐based instruction group for taught lesions (mean difference = 38%, 95% confidence interval [CI] 20–56; p < 0.001) and untaught lesions (mean difference = 31%, 95% CI 15–48; p < 0.001). Conclusions Schema‐based instruction was associated with improved retention of structured knowledge and diagnostic performance among novices. This study provides important proof‐of‐concept for a schema‐based approach and suggests there is substantial benefit to using this approach with novice trainees.     

The importance of seeing the patient: test-enhanced learning with standardized patients and written tests improves clinical application of knowledge

Previous research has shown that repeated retrieval with written tests produces superior long-term retention compared to repeated study. However, the degree to which this increased retention transfers to clinical application has not been investigated. In addition, increased retention obtained through written testing has not been compared to other forms of testing, such as simulation testing with a standardized patient (SP). In our study, 41 medical students learned three clinical topics through three different learning activities: testing with SPs, testing using written tests, and studying a review sheet. Students were randomized in a counter-balanced fashion to engage in one learning activity per topic. They participated in four weekly testing/studying sessions to learn the material, engaging in the same activity for a given topic in each session. Six months after initial learning, they returned to take an SP test on each topic, followed by a written test on each topic 1 week later. On both forms of final testing, we found that learning through SP testing and written testing generally produced superior long-term retention compared to studying a review sheet. SP testing led to significantly better performance on the final SP test relative to written testing, but there was no significant difference between the two testing conditions on the final written test. Overall, our study shows that repeated retrieval practice with both SPs and written testing enhances long-term retention and transfer of knowledge to a simulated clinical application.     

Test-enhanced learning in medical education

CONTEXT: In education, tests are primarily used for assessment, thus permitting teachers to assess the efficacy of their curriculum and to assign grades. However, research in cognitive psychology has shown that tests can also directly affect learning by promoting better retention of information, a phenomenon known as the testing effect. COGNITIVE PSYCHOLOGY RESEARCH: Cognitive psychology laboratory studies show that repeated testing of information produces superior retention relative to repeated study, especially when testing is spaced out over time. Tests that require effortful retrieval of information, such as short-answer tests, promote better retention than tests that require recognition, such as multiple-choice tests. The mnemonic benefits of testing are further enhanced by feedback, which helps students to correct errors and confirm correct answers. APPLICATION TO MEDICAL EDUCATION: Medical educational research has focused extensively on assessment issues. Such assessment research permits the conclusion that clinical expertise is founded on a broad fund of knowledge and effective memory networks that allow easy access to that knowledge. Test-enhanced learning can potentially strengthen clinical knowledge that will lead to improved expertise. CONCLUSIONS: Tests should be given often and spaced out in time to promote better retention of information. Questions that require effortful recall produce the greatest gains in memory. Feedback is crucial to learning from tests. Test-enhanced learning may be an effective tool for medical educators to use in promoting retention of clinical knowledge.     

Differential Effects of Two Types of Formative Assessment in Predicting Performance of First-year Medical Students

Formative assessments are systematically designed instructional interventions to assess and provide feedback on students’ strengths and weaknesses in the course of teaching and learning. Despite their known benefits to student attitudes and learning, medical school curricula have been slow to integrate such assessments into the curriculum. This study investigates how performance on two different modes of formative assessment relate to each other and to performance on summative assessments in an integrated, medical-school environment. Two types of formative assessment were administered to 146 first-year medical students each week over 8 weeks: a timed, closed-book component to assess factual recall and image recognition, and an un-timed, open-book component to assess higher order reasoning including the ability to identify and access appropriate resources and to integrate and apply knowledge. Analogous summative assessments were administered in the ninth week. Models relating formative and summative assessment performance were tested using Structural Equation Modeling. Two latent variables underlying achievement on formative and summative assessments could be identified; a “formative-assessment factor” and a “summative-assessment factor,” with the former predicting the latter. A latent variable underlying achievement on open-book formative assessments was highly predictive of achievement on both open- and closed-book summative assessments, whereas a latent variable underlying closed-book assessments only predicted performance on the closed-book summative assessment. Formative assessments can be used as effective predictive tools of summative performance in medical school. Open-book, un-timed assessments of higher order processes appeared to be better predictors of overall summative performance than closed-book, timed assessments of factual recall and image recognition. This research was presented at the 86th Annual Meeting of the American Educational Research Association (AERA) in Montreal, Canada, April 11–15, 2005.     

Test-enhanced learning may be a gender-related phenomenon explained by changes in cortisol level

Medical Education 2011: 45 : 192–199 Context Testing increases memory of a topic studied more than additional study or training. The mechanisms by which this occurs are not clearly understood. Testing can be stressful and studies suggest that the stress hormone cortisol has modulating effects on memory, predominantly in men. The aim of this study was to investigate whether cardiopulmonary resuscitation (CPR) skills testing induce a cortisol increase, whether the cortisol increase enhances retention of CPR skills, and how this relates to gender. Methods We randomised a convenience sample of medical students attending a mandatory course to one intervention and one control group. Students received a 4‐hour course on CPR skills. During the final half‐hour of the intervention course, participants were tested in CPR scenarios, whereas the control group underwent additional training. We assessed learning outcomes 2 weeks later by rating student performance in a CPR scenario using a checklist and a single blinded assessor. We measured salivary cortisol pre‐course, half an hour before the end of the course and post‐course, and compared learning outcomes and cortisol responses between groups and genders. Results In total, 146 of 202 (72%) students completed the study. We found a significant difference in learning outcome between the intervention and control groups for both genders (mean ± standard deviation, 5.0 ± 3.5; p = 0.006). We found a significant effect of increase in cortisol on learning outcome in men. The correlation between learning outcome and cortisol increase was medium to large for men (r = 0.38), but not for women (r = ? 0.05). Conclusions Cardiopulmonary resuscitation skills testing induces a rise in cortisol in men, which is related to the better retention of skills in men. Cortisol modulates test‐enhanced learning in men.     

Development of knowledge in basic sciences: a comparison of two medical curricula

Medical Education 2012: 46 : 1206–1214 Context Basic medical sciences education differs among medicine courses, especially as traditional and integrated problem‐based learning (PBL) curricula teach basic sciences in very different ways. The literature shows no clear differences in the performance of students of these different educational philosophies. The Charité Medical University of Berlin (Charité Universitätsmedizin Berlin) teaches both a traditional medical curriculum (TMC) and a PBL reformed medical curriculum (RMC). Both curricula conduct the Progress Test in Medicine (PTM), which examines competence in the basic and clinical sciences from the first to the last semester. Objectives The aim of this study was to compare the development and retention of knowledge in the basic medical sciences between students on the traditional and reformed undergraduate medical curricula, respectively. Methods For each student and single PTM, relative frequencies of correct answers were computed for basic sciences items only and for the whole curriculum. Frequencies were averaged and grouped by semester and curriculum. Analyses of variance (anova s) were performed at all measurement points with a Bonferroni‐corrected p‐value at the level of p < 0.005. Eta‐squared (η²) was used to classify effect size. Results In the first three semesters, RMC students slightly outperform TMC students in the basic sciences, although TMC students receive more systematic teaching. After this, TMC students develop a peak of knowledge in basic sciences and overtake RMC students. The knowledge of TMC students then decreases over time, but despite this, they perform better in the final semester. Students on the RMC show constant progress throughout their undergraduate studies. Overall, the development of medical knowledge is consistent in both curricula. There is no significant difference in this outcome between the traditional and PBL courses. Conclusions Progress testing as a longitudinal method allows us to better understand the development of knowledge during formal undergraduate education. The main difference between traditional and problem‐based medical education seems to be provoked by the high‐stakes national examination undertaken in the traditional course (the Physikum).     

How Can I Know What I Don't Know? Poor Self Assessment in a Well-Defined Domain

As the rapidity with which medical knowledge is generated and disseminated becomes amplified, an increasing emphasis has been placed on the need for physicians to develop the skills necessary for life-long learning. One such skill is the ability to evaluate one's own deficiencies. A ubiquitous finding in the study of self-assessment, however, is that self-ratings are poorly correlated with other performance measures. Still, many educators view the ability to recognize and communicate one's deficiencies as an important component of adult learning. As a result, two studies have been performed in an attempt to improve upon this status quo. First, we tried to re-define the limits within which self-assessments should be used, using Rosenblit and Keil's argument that calibration between perceived and actual performance will be better within taxonomies that are regularly tested (e.g., factual knowledge) compared to those that are not (e.g., conceptual knowledge). Second, we tried to norm reference individuals based on both the performance of their colleagues and their own historical performance on McMaster's Personal Progress Inventory (a multiple choice question test of medical knowledge). While it appears that students are able to (a) make macro-level self-assessments (i.e., to recognize that third year students typically outperform first year students), and (b) judge their performance relatively accurately after the fact, students were unable to predict the percentage of questions they would answer correctly with a testing procedure in which they have had a substantial amount of feedback. Previous test score was a much better predictor of current test performance than were individuals' expectations.     

Spaced education improves the retention of clinical knowledge by medical students: a randomised controlled trial

Purpose Medical knowledge learned by trainees is often quickly forgotten. How can the educational process be tailored to shift learning into longer‐term memory? We investigated whether ‘spaced education’, consisting of weekly e‐mailed case scenarios and clinical questions, could improve the retention of students' learning. Methods During the 2004–5 surgery clerkships, 3rd‐year students completed a mandatory 1‐week clinical rotation in urology and validated web‐based teaching programme on 4 core urology topics. Spaced educational e‐mails were constructed on all 4 topics based on a validated urology curriculum. Each consisted of a short clinically relevant question or clinical case scenario in multiple‐choice question format, followed by the answer, teaching point summary and explanations of the answers. Students were randomised to receive weekly e‐mailed case scenarios in only 2 of the 4 urology topics upon completion of their urology rotation. Students completed a validated 28‐item test (Cronbach's α = 0.76) on all 4 topics prior to and after the rotation and at the end of the academic year. Results A total of 95 of 133 students (71%) completed the end‐of‐year test. There were no significant differences in baseline characteristics between randomised cohorts. Spaced education significantly improved composite end‐of‐year test scores (P < 0.001, paired t‐test). The impact of the spaced educational e‐mails was largest for those students who completed their urology education 6–8 and 9–11 months previously (Cohen's effect sizes of 1.01 and 0.73, respectively). Conclusion Spaced education consisting of clinical scenarios and questions distributed weekly via e‐mail can significantly improve students' retention of medical knowledge.     

Influences of deep learning,need for cognition and preparation time on open‐ and closed‐book test performance

Medical Education 2010: 44 : 884–891 Objectives The ability to master discipline‐specific knowledge is one of the competencies medical students must acquire. In this context, ‘mastering’ means being able to recall and apply knowledge. A way to assess this competency is to use both open‐ and closed‐book tests. Student performance on both tests can be influenced by the way the student processes information. Deep information processing is expected to influence performance positively. The personal preferences of students in relation to how they process information in general (i.e. their level of need for cognition) may also be of importance. In this study, we examined the inter‐relatedness of deep learning, need for cognition and preparation time, and scores on open‐ and closed‐book tests. Methods This study was conducted at the University Medical Centre Groningen. Participants were Year 2 students (n = 423). They were asked to complete a questionnaire on deep information processing, a scale for need for cognition on a questionnaire on intellectualism and, additionally, to write down the time they spent on test preparation. We related these measures to the students’ scores on two tests, both consisting of open‐ and closed‐book components and used structural equation modelling to analyse the data. Results Both questionnaires were completed by 239 students (57%). The results showed that need for cognition positively influenced both open‐ and closed‐book test scores (β‐coefficients 0.05 and 0.11, respectively). Furthermore, study outcomes measured by open‐book tests predicted closed‐book test results better than the other way around (β‐coefficients 0.72 and 0.11, respectively). Conclusions Students with a high need for cognition performed better on open‐ as well as closed‐book tests. Deep learning did not influence their performance. Adding open‐book tests to the regularly used closed‐book tests seems to improve the recall of knowledge that has to be known by heart. Need for cognition may provide a valuable addition to existing theories on learning.     

Significant increase in factual knowledge with web-assisted problem-based learning as part of an undergraduate cardio-respiratory curriculum

In recent years, increasing attention has been paid to web-based learning although the advantages of computer-aided instruction over traditional teaching formats still need to be confirmed. This study examined whether participation in an online module on the differential diagnosis of dyspnoea impacts on student performance in a multiple choice examination of factual knowledge in cardiology and pneumology. A virtual problem-based learning environment for medical students supervised by postgraduate teachers was created. Seventy-four out of 183 fourth-year medical students volunteered to use the online module while attending a 6-week cardio-respiratory curriculum in summer 2007. Of these, 40 were randomly selected to be included (intervention group); the remaining 34 served as an internal control group. Analysis of all written exams taken during the preceding term showed that both groups were comparable (86.4 ± 1.1 vs. 85.9 ± 1.1%; p = 0.751). Students in the intervention group scored significantly higher in the final course assessment than students allocated to the control group (84.8 ± 1.3 vs. 79.5 ± 1.4%; p = 0.006; effect size 0.67). Thus, additional problem-based learning with an online module as part of an undergraduate cardio-respiratory curriculum lead to higher students’ scores in an exam testing factual knowledge. Whether using this teaching format increases overall student motivation to engage in the learning process needs to be further investigated.     

Techniques for teaching electrocardiogram interpretation: self-directed learning is less effective than a workshop or lecture

Medical Education 2011: 45 : 347–353 Context Teaching 12‐lead electrocardiogram (ECG) interpretation to students and residents is a challenge for medical educators. To date, few studies have compared the effectiveness of different techniques used for ECG teaching. Objectives This study aimed to determine if common teaching techniques, such as those involving workshops, lectures and self‐directed learning (SDL), increase medical students’ ability to correctly interpret ECGs. It also aimed to compare the effectiveness of these formats. Methods This was a prospective randomised study conducted over a 28‐month period. Year 4 medical students were randomised to receive teaching in ECG interpretation using one of three teaching formats: workshop, lecture or SDL. All three formats covered the same content. Students were administered three tests: a pre‐test (before teaching); a post‐test (immediately after teaching), and a retention test (1 week after teaching). Each tested the same content using 25 questions worth 1 point each. A mixed‐model repeated‐measures analysis of variance (anova ) with least squares post hoc analysis was conducted to determine if differences in test scores between the formats were statistically significant. Results Of the 223 students for whom data were analysed, 79 were randomised to a workshop, 82 to a lecture‐based format and 62 to SDL. All three teaching formats resulted in a statistically significant improvement in individual test scores (p < 0.001). Comparison of the lecture‐ and workshop‐based formats demonstrated no difference in test scores (marginal mean [MM] for both formats = 12.4, 95% confidence interval [95% CI] 11.7–13.2]; p = 0.99). Test scores of students using SDL (MM = 10.7, 95% CI 9.8–11.5) were lower than those of students in the workshop (p = 0.003) and lecture (p = 0.002) groups. Conclusions Compared with those taught using workshop‐ and lecture‐based formats, medical students learning ECG interpretation by SDL had lower test scores.     

Repeated testing improves long‐term retention relative to repeated study: a randomised controlled trial

Context Laboratory studies in cognitive psychology with relatively brief final recall intervals suggest that repeated retrieval in the form of tests may result in better retention of information compared with repeated study. Objectives Our study evaluates if repeated testing of material taught in a real‐life educational setting (a didactic conference for paediatric and emergency medicine residents) replicates these findings when measured at a more educationally relevant final recall interval of 6 months. Methods Residents participated in an interactive teaching session on two topics: (i) status epilepticus, and (ii) myasthenia gravis. Residents were randomised to two counter‐balanced groups which either took tests on status epilepticus and studied a review sheet on myasthenia gravis (SE‐T/MG‐S group) or took tests on myasthenia gravis and studied a review sheet on status epilepticus (MG‐T/SE‐S group). Testing and studying occurred immediately after teaching and then at two additional times at intervals of about 2 weeks. Residents received feedback after each test. Tests consisted of short‐answer questions and the review sheets consisted of information identical to that on the answer sheets for the tests. At about 6 months residents took a final test on both topics. Results Nineteen residents in the SE‐T/MG‐S group and 21 residents in the MG‐T/SE‐S group completed the study. Collapsing across groups, repeated testing produced final test scores that were an average of 13% higher than those produced by repeated study (39% versus 26%) at > 6 months after the initial teaching session (t[78] = 3.93, standard error of the difference = 0.03, P < 0.001, d = 0.91). Conclusions Repeated testing with feedback appears to result in significantly greater long‐term retention of information taught in a didactic conference than repeated, spaced study. Testing should be considered for its potential impact on learning and not only as an assessment device.     

The effectiveness and reliability of peer-marking in first-year medical students

BACKGROUND: Peer-marking has been suggested as a method to enhance self-directed learning and reflection, although whether this improves performance is unclear. This study evaluated the impact of peer-marking on examination performance and investigated its reliability and acceptability to students. METHODS: First-year medical students were randomised to peer-marking using a model answer or no intervention (control arm). Student scores were compared with tutor-marked scores. Two months later, students completed a summative assessment and performance was compared between students randomised to peer-marking and the control arm. A focus group was held with students in the intervention arm to capture their experiences and attitudes. RESULTS: A total of 289 of 568 students consented to participate and 147 were randomised to peer-marking (142 controls). Students randomised to peer-marking achieved marginally higher examination marks (1.5% difference, 95% CI -0.8% to 3.9%, P = 0.19) than controls (adjusting for year and in-course assessment), although this may have been due to chance. Students were harsher markers than the tutors. Focus group analysis suggested that students valued peer-marking, although concerns about passing judgement on a colleague's work were expressed. CONCLUSIONS: Peer-marking did not have a substantial effect on examination performance, although a modest effect cannot be excluded. Students gained insight into examination technique but may not have gained deeper knowledge. Given its potential positive educational value, further work is required to understand how peer-marking can be used more effectively to enhance the learning experience.     

Learning during simulation training is prone to retroactive interference

Medical Education 2012: 46 : 299–305 Context Retroactive interference occurs when newly acquired information inhibits recall of previously learned information. This has been shown to influence recall of sounds, tastes and word associations, and is typically seen when learners receive training on one area of content and are then exposed to new content before being evaluated on the original content. Thus far, retroactive interference has received little attention in medical education and has not been studied during simulation training. Our objective was to evaluate whether retroactive interference occurs during simulation training. Methods We randomised 167 Year 1 medical students to one of two training protocols. After training on a cardiac murmur, participants were tested either on the same cardiac murmur followed by a novel murmur (the non‐interference protocol), or on the novel murmur followed by the training murmur (the interference protocol). We evaluated performance on both murmurs at 1 hour and 6 weeks post‐training. Results We found a significant interaction between training protocol and diagnostic performance on training versus novel murmurs at both testing time‐points. Students in the non‐interference protocol had increased odds of achieving success on the training murmur relative to the novel murmur at 1 hour (odds ratio [OR] 4.96; p < 0.001) and at 6 weeks (OR 4.23; p = 0.001) after training. By comparison, students in the interference protocol did not demonstrate improved performance on the training murmur relative to the novel murmur at either evaluation (1 hour post‐training: OR 0.56 [p = 0.08]; 6 weeks post‐training: OR 0.66 [p = 0.23]). Conclusions Consistent with the theory of retroactive interference, students who encountered a novel murmur between training and evaluation on the murmur on which they had been trained showed no improvement in diagnostic performance following simulation training. These findings should serve to warn educators to consider retroactive interference when designing simulation training sessions.     

Applying multimedia design principles enhances learning in medical education

Medical Education 2011: 45 : 818–826 Context The Association of American Medical Colleges’ Institute for Improving Medical Education’s report entitled ‘Effective Use of Educational Technology’ called on researchers to study the effectiveness of multimedia design principles. These principles were empirically shown to result in superior learning when used with college students in laboratory studies, but have not been studied with undergraduate medical students as participants. Methods A pre‐test/post‐test control group design was used, in which the traditional‐learning group received a lecture on shock using traditionally designed slides and the modified‐design group received the same lecture using slides modified in accord with Mayer’s principles of multimedia design. Participants included Year 3 medical students at a private, midwestern medical school progressing through their surgery clerkship during the academic year 2009–2010. The medical school divides students into four groups; each group attends the surgery clerkship during one of the four quarters of the academic year. Students in the second and third quarters served as the modified‐design group (n = 91) and students in the fourth‐quarter clerkship served as the traditional‐design group (n = 39). Results Both student cohorts had similar levels of pre‐lecture knowledge. Both groups showed significant improvements in retention (p < 0.0001), transfer (p < 0.05) and total scores (p < 0.0001) between the pre‐ and post‐tests. Repeated‐measures anova analysis showed statistically significant greater improvements in retention (F = 10.2, p = 0.0016) and total scores (F = 7.13, p = 0.0081) for those students instructed using principles of multimedia design compared with those instructed using the traditional design. Conclusions Multimedia design principles are easy to implement and result in improved short‐term retention among medical students, but empirical research is still needed to determine how these principles affect transfer of learning. Further research on applying the principles of multimedia design to medical education is needed to verify the impact it has on the long‐term learning of medical students, as well as its impact on other forms of multimedia instructional programmes used in the education of medical students.     

The testing effect on skills learning might last 6 months

In a recent study we found that testing as a final activity in a skills course increases the learning outcome compared to spending an equal amount of time practicing. Whether this testing effect measured as skills performance can be demonstrated on long-term basis is not known. The research question was: does testing as a final activity in a cardio-pulmonary resuscitation (CPR) skills course increase learning outcome when assessed after half a year, compared to spending an equal amount of time practicing? The study was an assessor-blinded randomised controlled trial. A convenient sample of 7th semester medical students attending a mandatory CPR course was randomised to intervention course or control course. Participants were taught in small groups. The intervention course included 3.5 h skills training plus 30 min of skills testing. The practice-only control course lasted 4 h. Both groups were invited to a retention assessment of CPR skills half a year later. Participants included 89/180 (50%) of those invited to participate in the study. Mean performance score was 75.9 (SD 11.0) in the intervention group (N = 48) and 70.3 (SD 17.1) in the control group, effect size 0.4. The difference between groups was not statistically significant, P = 0.06. This study suggests that testing as a final activity in a CPR skills course might have an effect on long-term learning outcome compared to spending an equal amount of time practicing the skills. Although this difference was not statistically significant, the identified effect size of 0.4 can have important clinical and educational implications.     

Visualising the invisible: a network approach to reveal the informal social side of student learning

World-wide, universities in health sciences have transformed their curriculum to include collaborative learning and facilitate the students’ learning process. Interaction has been acknowledged to be the synergistic element in this learning context. However, students spend the majority of their time outside their classroom and interaction does not stop outside the classroom. Therefore we studied how informal social interaction influences student learning. Moreover, to explore what really matters in the students learning process, a model was tested how the generally known important constructs—prior performance, motivation and social integration—relate to informal social interaction and student learning. 301 undergraduate medical students participated in this cross-sectional quantitative study. Informal social interaction was assessed using self-reported surveys following the network approach. Students’ individual motivation, social integration and prior performance were assessed by the Academic Motivation Scale, the College Adaption Questionnaire and students’ GPA respectively. A factual knowledge test represented student’ learning. All social networks were positively associated with student learning significantly: friendships (β?=?0.11), providing information to other students (β?=?0.16), receiving information from other students (β?=?0.25). Structural equation modelling revealed a model in which social networks increased student learning (r?=?0.43), followed by prior performance (r?=?0.31). In contrast to prior literature, students’ academic motivation and social integration were not associated with students’ learning. Students’ informal social interaction is strongly associated with students’ learning. These findings underline the need to change our focus from the formal context (classroom) to the informal context to optimize student learning and deliver modern medics.     

What happens to the student? The neglected variable in educational outcome research

Disputes about the superiority of teaching methods often remain unresolved. The essential question we continuously want to answer is: Which teaching methods yield the best knowledge and skills in students? Abundant literature, in medical education and in education in general, on research with educational methods as independent variables and measures of outcome (e.g., test scores) as the dependent variable often point at "no significant difference" or only small differences between methods. Many factors do influence the educational outcome in students and large statistical power (such as meta analysis) should be helpful to eliminate many sources of error. However, one source we cannot tackle this way. That is, students will usually adapt quantity and quality of studying to meet testing requirements. In doing so, they may compensate for teaching quality. Some teaching may generate more effort in students than other teaching. Since test scores reflect primarily student activities, it is their efforts that may bring differences in teaching methods close to equality in test scores. Therefore, knowledge and skills should not be considered the primary outcome of teaching but the outcome of learning activities. If we want to discriminate between teaching methods, we must at least consider what happens to students.     

Very long-term retention of basic science knowledge in doctors after graduation

Medical Education 2011: 45: 422–430 Context Despite frequent complaints that biomedical knowledge is quickly forgotten after it has been learned, few investigations of actual long‐term retention of basic science knowledge have been conducted in the medical domain. Objectives Our aim was to illuminate the long‐term retention of basic science knowledge, particularly of unrehearsed knowledge. Methods Using a cross‐sectional study design, medical students and doctors in the Netherlands were tested for retention of basic science knowledge. Relationships between retention interval and proportion of correct answers on a knowledge test were investigated. Results The popular notion that most of basic science knowledge is forgotten shortly after graduation is not supported by our findings. With respect to the full test scores, which reflect a composite of unrehearsed and rehearsed knowledge, performance decreased from approximately 40% correct answers for students still in medical school, to 25–30% correct answers for doctors after many years of practice. When rehearsal during the retention interval is controlled for, it appears that little knowledge is lost for 1.5–2 years after it was last used; from then on, retention is best described by a negatively accelerated (logarithmic) forgetting curve. After ≥ 25 years, retention levels were in the range of 15–20%. Conclusions Conclusions about the forgetting of unrehearsed knowledge in this study are in line with findings reported in other domains: it proceeds in accordance with the Ebbinghaus curve for meaningful material, except that in our findings the ‘downward’ part appears to start later than in most other studies. The limitations of the study are discussed and possible ramifications for medical education are proposed.