Evaluating the validity evidence of an OSCE: results from a new medical school

Background

To prevent the problems of traditional clinical evaluation, the “Objective Structured Clinical Examination (OSCE)” was presented by Harden as a more valid and reliable assessment instrument. However, an essential condition to guarantee a high-quality and effective OSCE is the assurance of evidence to support the validity of its scores. This study examines the psychometric properties of OSCE scores, with an emphasis on consequential and internal structure validity evidence.

Methods

Fifty-three first year medical students took part in a summative OSCE at the Lebanese American University-School of Medicine. Evidence to support consequential validity was gathered by using criterion-based standard setting methods. Internal structure validity evidence was gathered by examining various psychometric measures both at the station level and across the complete OSCE.

Results

Compared to our actual method of computing results, the introduction of standard setting resulted in lower students’ average grades and a higher cut score. Across stations, Cronbach’s alpha was moderately low.

Conclusion

Gathering consequential and internal structure validity evidence by multiple metrics provides support for or against the quality of an OSCE. It is critical that this analysis be performed routinely on local iterations of given tests, and the results used to enhance the quality of assessment.

     

Temporal stability of objective structured clinical exams: a longitudinal study employing item response theory

Background

The objective structure clinical examination (OSCE) has been used since the early 1970s for assessing clinical competence. There are very few studies that have examined the psychometric stability of the stations that are used repeatedly with different samples. The purpose of the present study was to assess the stability of objective structured clinical exams (OSCEs) employing the same stations used over time but with a different sample of candidates, SPs, and examiners.

Methods

At Time 1, 191 candidates and at Time 2 (one year apart), 236 candidates participated in a 10-station OSCE; 6 of the same stations were used in both years. Generalizability analyses (Ep²) were conducted. Employing item response analyses, test characteristic curves (TCC) were derived for each of the 6 stations for a 2-parameter model. The TCCs were compared across the two years, Time 1 and 2.

Results

The Ep² of the OSCEs exceeded.70. Standardized thetas (θ) and discriminations were equivalent for the same station across the two year period indicating equivalent TCCs for a 2-parameter model.

Conclusion

The 6 OSCE stations used by the AIMG program over two years have adequate internal consistency reliability, stable generalizability (Ep²) and equivalent test characteristics. The process of assessment employed for IMG’s are stable OSCE stations that may be used several times over without compromising psychometric properties.With careful security, high-stakes OSCEs may use the same stations that have high internal consistency and generalizability repeatedly as the psychometric properties are stable over several years with different samples of candidates.

     

Fitness to practise sanctions in UK doctors are predicted by poor performance at MRCGP and MRCP(UK) assessments: data linkage study

Background

The predictive validity of postgraduate examinations, such as MRCGP and MRCP(UK) in the UK, is hard to assess, particularly for clinically relevant outcomes. The sanctions imposed on doctors by the UK’s General Medical Council (GMC), including erasure from the Medical Register, are indicators of serious problems with fitness to practise (FtP) that threaten patient safety or wellbeing. This data linkage study combined data on GMC sanctions with data on postgraduate examination performance.

Methods

Examination results were obtained for UK registered doctors taking the MRCGP Applied Knowledge Test (AKT; n?=?27,561) or Clinical Skills Assessment (CSA; n?=?17,365) at first attempt between 2010 and 2016 or taking MRCP(UK) Part 1 (MCQ; n?=?37,358), Part 2 (MCQ; n?=?28,285) or Practical Assessment of Clinical Examination Skills (PACES; n?=?27,040) at first attempt between 2001 and 2016. Exam data were linked with GMC actions on a doctor’s registration from September 2008 to January 2017, sanctions including Erasure, Suspension, Conditions on Practice, Undertakings or Warnings (ESCUW). Examination results were only considered at first attempts. Multiple logistic regression assessed the odds ratio for ESCUW in relation to examination results. Multiple imputation was used for structurally missing values.

Results

Doctors sanctioned by the GMC performed substantially less well on MRCGP and MRCP(UK), with a mean Cohen’s d across the five exams of ??0.68. Doctors on the 2.5th percentile of exam performance were about 12 times more likely to have FtP problems than those on the 97.5th percentile. Knowledge assessments and clinical assessments were independent predictors of future sanctions, with clinical assessments predicting ESCUW significantly better. The log odds of an FtP sanction were linearly related to examination marks over the entire range of performance, additional performance increments lowering the risk of FtP sanctions at all performance levels.

Conclusions

MRCGP and MRCP(UK) performance are valid predictors of professionally important outcomes that transcend simple knowledge or skills and the GMC puts under the headings of conduct and trust. Postgraduate examinations may predict FtP sanctions because the psychological processes involved in successfully studying, understanding and practising medicine at a high level share similar mechanisms to those underlying conduct and trust.

     

Headache-Specific Locus of Control and Migraine-Related Quality of Life: Understanding the Role of Anxiety

Purpose

This cross-sectional study examined the relationship between headache-specific locus of control (HSLC) and migraine-related quality of life, and anxiety as a mediator of this relationship.

Method

Two hundred and thirty-two people with migraine participated in the treatment of severe migraine trial. At baseline, participants completed self-report questionnaires of headache-specific locus of control (HSLC; subscales?=?internal, chance, and medical professionals), anxiety, and migraine-related quality of life. Correlations examined relationships between HSLC, anxiety, and migraine-related quality of life; ordinary least squares regression evaluated anxiety as a mediator of the relationship between HSLC and migraine-related quality of life.

Results

Higher internal HSLC was related to higher overall migraine-related quality of life (ps?<?.05) and emotion function impairments (p?=?.012). Anxiety mediated the relationship between internal HSLC and all measures of migraine-specific quality of life (ps?<?.05). Higher external (medical professionals and chance) HSLC was related to higher migraine-related quality of life impairments (all ps?<?.001).

Conclusion

All HSLC beliefs are associated with higher migraine-related quality of life impairments. Anxiety mediates the relationship between internal HSLC and migraine-related quality of life.

     

Towards a conceptual framework demonstrating the effectiveness of audiovisual patient descriptions (patient video cases): a review of the current literature

Background

Technological advances have enabled the widespread use of video cases via web-streaming and online download as an educational medium. The use of real subjects to demonstrate acute pathology should aid the education of health care professionals. However, the methodology by which this effect may be tested is not clear.

Methods

We undertook a literature review of major databases, found relevant articles relevant to using patient video cases as educational interventions, extracted the methodologies used and assessed these methods for internal and construct validity.

Results

A review of 2532 abstracts revealed 23 studies meeting the inclusion criteria and a final review of 18 of relevance. Medical students were the most commonly studied group (10 articles) with a spread of learner satisfaction, knowledge and behaviour tested. Only two of the studies fulfilled defined criteria on achieving internal and construct validity. The heterogeneity of articles meant it was not possible to perform any meta-analysis.

Conclusions

Previous studies have not well classified which facet of training or educational outcome the study is aiming to explore and had poor internal and construct validity. Future research should aim to validate a particular outcome measure, preferably by reproducing previous work rather than adopting new methods. In particular cognitive processing enhancement, demonstrated in a number of the medical student studies, should be tested at a postgraduate level.

     

Reliability of the Interprofessional Collaborator Assessment Rubric (ICAR) in Multi Source Feedback (MSF) with post-graduate medical residents

Background

Increased attention on collaboration and teamwork competency development in medical education has raised the need for valid and reliable approaches to the assessment of collaboration competencies in post-graduate medical education. The purpose of this study was to evaluate the reliability of a modified Interprofessional Collaborator Assessment Rubric (ICAR) in a multi-source feedback (MSF) process for assessing post-graduate medical residents’ collaborator competencies.

Methods

Post-graduate medical residents (n?=?16) received ICAR assessments from three different rater groups (physicians, nurses and allied health professionals) over a four-week rotation. Internal consistency, inter-rater reliability, inter-group differences and relationship between rater characteristics and ICAR scores were analyzed using Cronbach’s alpha, one-way and two-way repeated measures ANOVA, and logistic regression.

Results

Missing data decreased from 13.1% using daily assessments to 8.8% utilizing an MSF process, p?=?.032. High internal consistency measures were demonstrated for overall ICAR scores (α?=?.981) and individual assessment domains within the ICAR (α?=?.881 to .963). There were no significant differences between scores of physician, nurse, and allied health raters on collaborator competencies (F2,5?=?1.225, p?=?.297, η2?=?.016). Rater gender was the only significant factor influencing scores with female raters scoring residents significantly lower than male raters (6.12 v. 6.82; F1,5?=?7.184, p?=?.008, η 2?=?.045).

Conclusion

The study findings suggest that the use of the modified ICAR in a MSF assessment process could be a feasible and reliable assessment approach to providing formative feedback to post-graduate medical residents on collaborator competencies.

     

Building a bigger tent in point-of-care ultrasound education: a mixed-methods evaluation of interprofessional,near-peer teaching of internal medicine residents by sonography students

Background

Point-of-care-ultrasound (POCUS) training is expanding in undergraduate and graduate medical education, but lack of trained faculty is a major barrier. Two strategies that may help mitigate this obstacle are interprofessional education (IPE) and near-peer teaching. The objective of this study was to evaluate a POCUS course in which diagnostic medical sonography (DMS) students served as near-peer teachers for internal medicine residents (IMR) learning to perform abdominal sonography.

Methods

Prior to the IPE workshop, DMS students participated in a train-the-trainer session to practice teaching and communication skills via case-based simulation. DMS students then coached first-year IMR to perform POCUS examinations of the kidney, bladder, and gallbladder on live models. A mixed-methods evaluation of the interprofessional workshop included an objective structured clinical exam (OSCE), course evaluation, and qualitative analysis of focus group interviews.

Results

Twenty-four of 24 (100%) IMR completed the OSCE, averaging 97.7/107 points (91.3%) (SD 5.2). Course evaluations from IMR and DMS students were globally positive. Twenty three of 24 residents (96%) and 6/6 DMS students (100%) participated in focus group interviews. Qualitative analysis identified themes related to the learning environment, scanning technique, and suggestions for improvement. IMR felt the interprofessional training fostered a positive learning environment and that the experience complimented traditional faculty-led workshops. Both groups noted the importance of establishing mutual understanding of expectations and suggested future workshops have more dedicated time for DMS student demonstration of scanning technique.

Conclusion

An interprofessional, near-peer workshop was an effective strategy for teaching POCUS to IMR. This approach may allow broader adoption of POCUS in medical education, especially when faculty expertise is limited.

     

A modified evidence-based practice- knowledge,attitudes, behaviour and decisions/outcomes questionnaire is valid across multiple professions involved in pain management

Background

A validated and reliable instrument was developed to knowledge, attitudes and behaviours with respect to evidence-based practice (EBB-KABQ) in medical trainees but requires further adaptation and validation to be applied across different health professionals.

Methods

A modified 33-item evidence-based practice scale (EBP-KABQ) was developed to evaluate EBP perceptions and behaviors in clinicians. An international sample of 673 clinicians interested in treatment of pain (mean age?=?45 years, 48% occupational therapists/physical therapists, 25% had more than 5 years of clinical training) completed an online English version of the questionnaire and demographics. Scaling properties (internal consistency, floor/ceiling effects) and construct validity (association with EBP activities, comparator constructs) were examined. A confirmatory factor analysis was used to assess the 4-domain structure EBP knowledge, attitudes, behavior, outcomes/decisions).

Results

The EBP-KABQ scale demonstrated high internal consistency (Cronbach’s alpha?=?0.85), no evident floor/ceiling effects, and support for a priori construct validation hypotheses. A 4-factor structure provided the best fit statistics (CFI =0.89, TLI =0.86, and RMSEA?=?0.06).

Conclusions

The EBP-KABQ scale demonstrates promising psychometric properties in this sample. Areas for improvement are described.

     

Developing counseling skills through pre-recorded videos and role play: a pre- and post-intervention study in a Pakistani medical school

Background

Interactive methods like role play, recorded video scenarios and objective structured clinical exam (OSCE) are being regularly used to teach and assess communication skills of medical students in the western world. In developing countries however, they are still in the preliminary phases of execution in most institutes. Our study was conducted in a naïve under resourced setup to assess the impact of such teaching methodologies on the counseling skills of medical students.

Methods

Fifty four 4^th year MBBS students were identified to be evaluated for communication skills by trained facilitators in a pre-intervention OSCE. The same group of students was given a demonstration of ideal skill level by means of videos and role playing sessions in addition to real life interaction with patients during hospital and community rotations. A post-intervention evaluation was carried out six months later through OSCE and direct observation through structured checklist (DOS) in hospital and community settings. The combined and individual performance levels of these students were analyzed.

Results

There was a statistically significant difference in the communication skills of students when assessed in the post-intervention OSCE (p = 0.000). Individual post-intervention percentages of study participants displayed improvement as well (n = 45, p = 0.02). No difference was observed between the scores of male and female students when assessed for two specific competencies of antenatal care and breast feeding counseling (p = 0.11). The mean DOS (%) score of 12 randomly selected students was much lower as compared to the post-intervention (%) score but the difference between them was statistically non significant, a result that may have been affected by the small sample size as well as other factors that may come into play in real clinical settings and were not explored in this study (59.41 ± 7.8 against 82.43 ± 22.08, p = 0.88).

Conclusions

Videos and role play in combination with community and clinical exposure are effective modes of teaching counseling skills to medical students. They can be successfully utilized even in a limited resource setup, as demonstrated by our trial.

     

<Emphasis Type="Italic">Helicobacter bilis</Emphasis>-Associated Suppurative Cholangitis in a Patient with X-Linked Agammaglobulinemia

?

Helicobacter bilis is a commensal bacterium causing chronic hepatitis and colitis in mice. In humans, enterohepatic Helicobacter spp. are associated with chronic hepatobiliary diseases.

Purpose

We aimed at understanding the microbial etiology in a patient with X-linked agammaglobulinemia presenting with suppurative cholangitis.

Methods

16S rDNA PCR directly performed on a liver biopsy retrieved DNA of H. bilis.

Results

Clinical outcome resulted in the normalization of clinical and biological parameters under antibiotic treatment by a combination of ceftriaxone, metronidazole, and doxycyclin followed by a 2-week treatment with moxifloxacin and a 2-month treatment with azithromycin.

Conclusion

In conclusion, these data suggest a specific clinical and microbiological approach in patients with humoral deficiency in order to detect H. bilis hepatobiliary diseases.

     

Exploration of a possible relationship between examiner stringency and personality factors in clinical assessments: a pilot study

Background

The reliability of clinical examinations is known to vary considerably. Inter-examiner variability is a key source of this variability. Some examiners consistently give lower scores to some candidates compared to other examiners and vice versa – the ‘hawk- dove’ effect. Stable examiner characteristics, such as personality factors, may influence examiner stringency. We investigated whether examiner stringency is related to personality factors.

Methods

We recruited 12 examiners to view and score a video-recorded five station OSCE of six Year 1 undergraduate medical students at our institution. In addition examiners completed a validated personality questionnaire. Examiners’ markings were tested for statistically significant differences using non-parametric one way analysis of variance. The relationship between examiners’ markings and examiner personality factors was investigated using Spearman correlation coefficient.

Results

At each station there was a statistically significant difference between examiners markings, confirming the presence of inter-examiner variability. Correlation analysis showed no association between stringency and any of the five major personality factors. When we omitted an outlier examiner we found a statistically significant negative correlation between examiner stringency and openness to experience with a correlation coefficients (rho) of – 0.66 (p?=?0.03). Conversely there was a moderate positive correlation between examiner stringency and neuroticism with a correlation coefficient (rho) of 0.73 (p?=?0.01).

Conclusions

In this study we did not find any relationship between examiner stringency and examiner personality factors. However, following the elimination of an outlier examiner from the analysis, we found a significant relationship between examiner stringency and two of the big five personality factors (neuroticism and openness to experience). The significance of this outlier is not known. As this was a small pilot study we recommend further studies in this field to investigate if there is a relationship between examiner stringency in clinical assessments and personality factors.

     

Psychometric Properties of the Arabic Version of the Drug Use Disorders Identification Test (DUDIT) in Clinical,Prison Inmate,and Student Samples

Purpose

The study aimed to validate the Arabic version of the Drug Use Disorders Identification Test (DUDIT) by (1) assessing its factor structure, (2) determining structural validity, (3) evaluating item-total and inter-item correlation, and (4) assessing its predictive validity.

Method

The study population included 169 prison inmates, 51 patients with clinical diagnosis of substance used disorder, and 53 students (N = 273). All participants completed the self-report version of the Arabic DUDIT. After exploratory factor analysis, internal consistency of the Arabic DUDIT was determined and external validation was performed.

Results

Principal factor analysis showed that Arabic DUDIT exhibited only one factor, which explained 66.9% of the variance. Reliability based on Cronbach’s alpha was .95. When compared to the DSM-IV substance use disorder diagnosis in a clinical sample, DUDIT had an area under the curve (AUC) of .98, with a sensitivity of .98 and a specificity of .90.

Conclusion

The Arabic version of DUDIT is a valid and reliable tool for screening for drug use in Arabic-speaking countries.

     

Psychometric Properties and Normative Data for a Swedish Version of the Modern Health Worries Scale

Purpose

The modern health worries (MHW) scale was developed to assess individuals’ worries about aspects of modernity and technology affecting personal health. The aim of this study was to psychometrically evaluate a Swedish version of the MHW scale and to provide Swedish normative data.

Method

Data were collected as part of the Västerbotten Environmental Health Study, which has a random sample of 3406 Swedish adults (18–79 years).

Results

The Swedish version of the MHW scale showed excellent internal consistency and satisfactory convergent validity. A four-factor structure consistent with the original version was confirmed. The model showed invariance across age and sex. A slightly positively skewed and platykurtic distribution was found. Normative data for the general population and for combinations of specific age groups (young, middle aged, and elderly) and sex are presented.

Conclusion

The psychometric properties of the Swedish version of the MHW scale suggest that use of this instrument is appropriate for assessing worries about modernity in Swedish-speaking and similar populations. The scale now has the advantage of good normative data being available. MHW may hold importance for understanding and predicting the development of functional disorders, such as idiopathic environmental intolerance and other medically unexplained conditions.

     

Does undertaking an intercalated BSc influence first clinical year exam results at a London medical school?

Background

Intercalated BScs (iBScs) are an optional part of the medical school curriculum in many Universities. Does undertaking an iBSc influence subsequent student performance? Previous studies addressing this question have been flawed by iBSc students being highly selected. This study looks at data from medical students where there is a compulsory iBSc for non-graduates. Our aim was to see whether there was any difference in performance between students who took an iBSc before or after their third year (first clinical year) exams.

Methods

A multivariable analysis was performed to compare the third year results of students at one London medical school who had or had not completed their iBSc by the start of this year (n = 276). A general linear model was applied to adjust for differences between the two groups in terms of potential confounders (age, sex, nationality and baseline performance).

Results

The results of third year summative exams for 276 students were analysed (184 students with an iBSc and 92 without). Unadjusted analysis showed students who took an iBSc before their third year achieved significantly higher end of year marks than those who did not with a mean score difference of 4.4 (0.9 to 7.9 95% CI, p = 0.01). (overall mean score 238.4 "completed iBSc" students versus 234.0 "not completed", range 145.2 - 272.3 out of 300).There was however a significant difference between the two groups in their prior second year exam marks with those choosing to intercalate before their third year having higher marks. Adjusting for this, the difference in overall exam scores was no longer significant with a mean score difference of 1.4 (-4.9 to +7.7 95% CI, p = 0.66). (overall mean score 238.0 " completed iBSc" students versus 236.5 "not completed").

Conclusions

Once possible confounders are controlled for (age, sex, previous academic performance) undertaking an iBSc does not influence third year exam results. One explanation for this confounding in unadjusted results is that students who do better in their second year exams are more likely to take an iBSc before their third year.

     

Negative Stress Beliefs Predict Somatic Symptoms in Students Under Academic Stress

Purpose

Medically unexplained symptoms are abundantly present in the general population. Stress may lead to increased symptom reporting because of widespread beliefs that it is dangerous for one’s health. This study aimed at clarifying the role of stress beliefs in somatic symptom reporting using a quasi-experimental study design.

Methods

Two hundred sixteen German university students (60 % of an initial sample of 363) were examined at the beginning of the term (less stressful period) and at the end of the term (stressful period due to exams). Negative beliefs about stress at baseline were expected to predict somatic symptoms at follow-up.

Results

Negative beliefs about stress at baseline significantly predicted somatic symptoms at follow-up (β?=?0.16, p?=?.012), even when controlling for general strain, physical and mental health status, neuroticism, optimism, and somatosensory amplification.

Conclusions

Being convinced that “stress is bad for you” was prospectively associated with somatic symptoms during a stressful period. Further research in patients with medically unexplained conditions is warranted to corroborate these findings.

     

Validation of an Egyptian Arabic Version of Skindex-16 and Quality of Life Measurement in Egyptian Patients with Skin Disease

Purpose

To translate and cross-culturally adapt the Skindex-16, a brief skin-disease-specific QoL questionnaire, into Arabic for Egyptians and to evaluate its measurement properties in Egyptian patients with skin disease.

Method

Translation and cultural adaption were performed following guidelines for cross-cultural adaption of health-related quality of life measures. Subsequently, Skindex-16 was administered to 500 consecutive dermatological patients and 500 healthy persons for verification of its reliability and validity. Next, we examined the effect of skin disorders on QoL as well as influences of age, sex, socioeconomic level, education, and use of medication on skin disease-related QoL.

Results

The instrument showed high internal consistency reliability and good construct and content validity. The quality of life was most deteriorated in the emotional domain followed by symptoms domain (mean percentage score 55.5 ± 23.1 and 31.7 ± 26.8 respectively), while functioning was the least affected domain (mean percentage score 29.1 ± 26.8). Higher total Skindex-16 scores were observed for patients with psoriasis, urticaria, acne, and atopic dermatitis than those with vitiligo and alopecia areata. Educational level, social class, medication use, and family history influenced the impact of skin disease on overall QoL.

Conclusion

The adapted Egyptian Arabic version of Skindex-16 is valid and reliable; it showed that skin problems had a clinically significant impact on QoL in Egyptian patients.

     

Neural connections between the nervus intermedius and the facial and vestibulocochlear nerves in the cerebellopontine angle: an anatomic study

Purpose

Unexpected clinical outcomes following transection of single nerves of the internal acoustic meatus have been reported. Therefore, this study aimed to investigate interneural connections between the nervus intermedius and the adjacent nerves in the cerebellopontine angle.

Methods

On 100 cadaveric sides, dissections were made of the facial/vestibulocochlear complex in the cerebellopontine angle with special attention to the nervus intermedius and potential connections between this nerve and the adjacent facial or vestibulocochlear nerves.

Results

A nervus intermedius was identified on all but ten sides. Histologically confirmed neural connections were found between the nervus intermedius and either the facial or vestibulocochlear nerves on 34 % of sides. The mean diameter of these small interconnecting nerves was 0.1 mm. The fiber orientation of these nerves was usually oblique (anteromedial or posterolateral) in nature, but 13 connections traveled anteroposteriorly. Connecting fibers were single on 81 % of sides, doubled on 16 %, and tripled on 3 %, six sides had connections both with the facial nerve anteriorly and the vestibular nerves posteriorly. On 6.5 % of sides, a connection was between the nervus intermedius and cochlear nerve. For vestibular nerve connections with the nervus intermedius, 76 % were with the superior vestibular nerve and 24 % with the inferior vestibular nerve.

Conclusions

Knowledge of the possible neural interconnections found between the nervus intermedius and surrounding nerves may prove useful to surgeons who operate in these regions so that inadvertent traction or transection is avoided. Additionally, unanticipated clinical presentations and exams following surgery may be due to such neural interconnections.

     

Pleiotropic regulations of neutrophil receptors response to sepsis

Introduction

Sepsis is a complex clinical condition that causes a high mortality rate worldwide. Numerous studies on the pathophysiology of sepsis have revealed an imbalance in the inflammatory network, thus leading to tissue damage, organ failure, and ultimately death. The impairment of neu­trophil migration is associated with the outcome of sepsis.

Methods

Literature review was performed on the roles of neutrophil recruitment and neutrophil receptors as pleiotropic regulators during sepsis. Additionally, we systematically classify neutrophil receptors with regard to the neutrophil response during sepsis and discuss the clinical implications of these receptors for the treatment of sepsis.

Results

Increasing evidence suggests that there is significant dysfunction in neutrophil recruitment during sepsis, characterized by the failure to migrate to the site of infection. Neutrophil receptors, as pleiotropic regulators, play important roles in the neutrophil response during sepsis.

Conclusions

Neutrophil receptors play key roles in chemotactic neutrophil migration and may prove to be suitable targets in future pharmacological therapies for sepsis.

     

Long-term follow up of factual knowledge after a single,randomised problem-based learning course

Background

The long-term effect of problem-based learning (PBL) on factual knowledge is poorly investigated. We took advantage of a previous randomised comparison between PBL and traditional teaching in a 3^rd year course to follow up factual knowledge of the students during their 4^th and 5^th year of medical school training.

Methods

3^rd year medical students were initially randomized to participate in a problem-based (PBL, n = 55), or a lecture-based (LBL, n = 57) course in basic pharmacology. Summative exam results were monitored 18 months later (after finishing a lecture-based course in clinical pharmacology). Additional results of an unscheduled, formative exam were obtained 27 months after completion of the first course.

Results

Of the initial sample of 112 students, 90 participated in the second course and exam (n = 45, 45). 32 (n = 17 PBL, n = 15 LBL) could be exposed to the third, formative exam. Mean scores (± SD) were 22.4 ± 6.0, 27.4 ± 4.9 and 20.1 ± 5.0 (PBL), or 22.2 ± 6.0, 28.4 ± 5.1 and 19.0 ± 4.7 (LBL) in the first, second and third test, respectively (maximum score: 40). No significant differences were found between the two groups.

Conclusion

A small-scale exposure to PBL, applied under randomized conditions but in the context of a traditional curriculum, does not sizeably change long-term presence of factual knowledge within the same discipline.