Longterm effects of problem-based learning: a comparison of competencies acquired by graduates of a problem-based and a conventional medical school

BACKGROUND: Problem-based learning (PBL) as an approach to the instruction of medical students has attracted much attention in recent years. However, its effect on the performance of its graduates is the subject of considerable debate. This article presents data from a large-scale study among graduates of a problem-based medical school and those of a conventional medical school to contribute to this discussion. PURPOSE: To study the longterm effects of problem-based medical training on the professional competencies of graduates. METHODS: A questionnaire was sent to all graduates since 1980 of a problem-based and a conventional medical school. Participants were requested to rate themselves on 18 professional competencies derived from the literature. RESULTS: The graduates of the PBL school scored higher on 14 of 18 professional competencies. Graduates of the problem-based school rated themselves as having much better interpersonal skills, better competencies in problem solving, self-directed learning and information gathering, and somewhat better task-supporting skills, such as the ability to work and plan efficiently. There were no sizeable differences with regard to general academic competencies, such as conducting research or writing a paper. Graduates from the conventional school rated themselves as having slightly more medical knowledge. The findings were shown to be valid and robust against possible response bias. CONCLUSION: The findings suggest that PBL not only affects the typical PBL-related competencies in the interpersonal and cognitive domains, but also the more general work-related skills that are deemed important for success in professional practice.     

Does problem‐based learning lead to deficiencies in basic science knowledge? An empirical case on anatomy

INTRODUCTION: Problem-based learning (PBL) is supposed to enhance the integration of basic and clinical sciences. In a non-integrative curriculum, these disciplines are generally taught in separate courses. Problem-based learning students perceive deficiencies in their knowledge of basic sciences, particularly in important areas such as anatomy. Outcome studies on PBL show controversial results, sometimes indicating that medical students at PBL schools have less knowledge of basic sciences than do their colleagues at more traditional medical schools. We aimed to identify differences between PBL and non-PBL students in perceived and actual levels of knowledge of anatomy. METHODS: Samples of Year 4 students in all eight medical schools in the Netherlands completed a questionnaire on perceived knowledge and took part in a computerised anatomy test consisting of both clinically contextualised items and items without context. RESULTS: Problem-based learning students were found to have the same perceived level of anatomy knowledge as students at other medical schools. Differences in actual levels of knowledge were found between schools. No significant effects on knowledge levels were found for PBL schools versus non-PBL schools. CONCLUSION: The results of this study show that PBL does not result in a lower level of anatomy knowledge than more traditional educational approaches. It remains to be ascertained whether the levels students attain are adequate. Subjects for further study are the desired level of anatomy knowledge at the end of undergraduate medical education and the effectiveness of basic science learning within a clinical context and with repetition over the course of the curriculum.     

Communication skills competencies: definitions and a teaching toolbox

BACKGROUND: Doctors' interpersonal and communication skills correlate with improved health care outcomes. International medical organisations require competency in communication skills. The Accreditation Council for Graduate Medical Education (ACGME) developed a toolbox for assessing this competency and 5 others, yet none initially for teaching these skills. PURPOSE AND METHODS: The original focus in the development of the ACGME competencies was evaluation. This paper represents a significant step toward defining methods for teaching communication skills competencies. A total of 16 medical education leaders from medical schools worldwide, participating in the 2003 Harvard Macy Institute Program for Physician Educators, worked together to: (1) further define the ACGME competency in interpersonal and communication skills; (2) delineate teaching strategies for each level of medical education; and (3) create a teaching toolbox to integrate communication skills competencies into medical curricula. Four subgroups defined subcompetencies, identified teaching strategies for undergraduate, graduate and postgraduate medical training and brought their work to the larger group. The expanded communication competencies and teaching strategies were determined by a consensus of the larger group, presented to 80 Harvard Macy Scholars and Faculty for further discussion, then finalised by consensus. CONCLUSION: The teaching toolbox expands the ACGME core communication competencies, adds 20 subcompetencies and connects these competencies to teaching strategies at each level of medical training. It represents the collaboration and consensus of a diverse international group of medical education leaders in a variety of medical specialities and institutions, all involved in teaching communication skills. The toolbox is applicable globally across different settings and specialities, and is sensitive to different definitions of health care.     

Effects of conventional and problem-based learning on clinical and general competencies and career development

Objective  To test hypotheses regarding the longitudinal effects of problem-based learning (PBL) and conventional learning relating to students' appreciation of the curriculum, self-assessment of general competencies, summative assessment of clinical competence and indicators of career development.
Methods  The study group included 2 complete cohorts of graduates who were admitted to the medical curriculum in 1992 (conventional curriculum, n  =   175) and 1993 (PBL curriculum, n  =   169) at the Faculty of Medicine, University of Groningen, the Netherlands. Data were obtained from student records, graduates' self-ratings and a literature search. Gender and secondary school grade point average (GPA) scores were included as moderator variables. Data were analysed by a stepwise multiple and logistic regression analysis.
Results  Graduates of the PBL curriculum scored higher on self-rated competencies. Contrary to expectations, graduates of the PBL curriculum did not show more appreciation of their curriculum than graduates of the conventional curriculum and no differences were found on clinical competence. Graduates of the conventional curriculum needed less time to find a postgraduate training place. No differences were found for scientific activities such as reading scientific articles and publishing in peer- reviewed journals. Women performed better on clinical competence than did men. Grade point average did not affect any of the variables.
Conclusions  The results suggest that PBL affects self-rated competencies. These outcomes confirm earlier findings. However, clinical competence measures did not support this finding.     

Perceptions of dissection by students in one medical school: beyond learning about anatomy. A qualitative study

INTRODUCTION: The practice of dissection, as part of undergraduate medical education, has recently resurfaced in the public eye. This paper focuses on a number of important learning outcomes that were reported by Year 1-5 medical students in a British medical school, during the dissection sessions in the first 2 years of their training, as part of a wider qualitative research project into undergraduate medical education. METHODS: A group of 29 students was selected by quota sampling, using the whole student population of the medical school as the sampling frame. Qualitative data were collected by 1 : 1 interviews with students and from formal non-participatory observations of dissection sessions. RESULTS: Apart from learning to cope with the overt 'emotional confrontation' with the cadavers which assists anatomical learning, 7 additional covert learning outcomes were identified by the students: teamwork, respect for the body, familiarisation of the body, application of practical skills, integration of theory and practice, preparation for clinical work, and appreciation of the status of dissection within the history of medicine. DISCUSSION: A number of medical schools have either removed the practical, hands-on aspect of dissection in the medical undergraduate curriculum or are seriously considering such a measure, on financial and/or human resource grounds. This study highlights the fact that dissection can impart anatomical knowledge as well as offer other relevant, positive learning opportunities to enhance the skills and attitudes of future doctors.     

Can undergraduate education have an effect on the ways in which pre‐registration house officers conceptualise communication?

AIMS: In 1994 Manchester Medical School introduced a learner-centred course using problem-based learning (PBL), which emphasises effective communication skills. This study explored how 2 cohorts of graduates conceptualised communication within their role as pre-registration house officer (PRHOs). METHODS: Graduates from the last year of the traditional and first year of the new course were interviewed 3 months after starting work. They were asked how well their courses had prepared them for working as PRHOs and were given a specific question about communication skills. Interviews were tape-recorded, transcribed, coded and analysed. RESULTS: In all, 24 traditional course graduates (TCGs) and 23 new course graduates (NCGs) were interviewed. Traditional course graduates tended to conceptualise communication as informing people rather than as involving negotiation or having therapeutic effects. Most TCGs considered good communicators 'were born that way' and did not think communication skills could be learned. Many NCGs had a richer concept of communication that recognised communication has therapeutic benefits for patients and involved negotiation. They understood it was possible to teach communication skills. However, a minority of NCGs conceptualised communication in a similar way to TCGs. CONCLUSION: Our study suggests that the new course has helped NCGs acquire a more complex concept of communication in their role as PRHOs.     

Variation by medical school in career choices of UK graduates of 1999 and 2000

OBJECTIVE: To report on how newly qualified doctors' specialty choices, and factors that influenced them, varied by medical school. DESIGN: Postal questionnaires. SETTING: United Kingdom. SUBJECTS: All doctors who graduated in the UK in 1999 and 2000. MAIN OUTCOME MEASURES: Choices of eventual career expressed 1 year after graduating, and factors influencing their choices. RESULTS: There were some significant differences between medical schools in the career choices made by their graduates. For example, the percentage of respondents who expressed the choice of general practice was significantly low among graduates of Oxford and Cambridge and high among graduates of Birmingham and Leicester. There was also significant variation between medical schools in choices for hospital medical and surgical specialties. There were significant differences, too, between medical schools in the extent to which career choices had been strongly influenced by graduates' inclinations before starting medical school and by their experience of their chosen specialty, particular teachers and departments at medical school. As well as the differences, however, there were also many similarities between the schools in graduates' career choices. DISCUSSION: Medical schools currently provide students with a broad training suited to any subsequent choice of specialty. We suggest that the similarities between schools in the career choices made by graduates are generally more striking than the differences. We raise the question of whether there should be any specialisation by individual schools to train students for careers in particular branches of medical practice.     

Educational supervisors evaluate the preparedness of graduates from a reformed UK curriculum to work as pre-registration house officers (PRHOs): a qualitative study

INTRODUCTION: In 1996 the University of Liverpool introduced an integrated problem-based learning (PBL) medical curriculum incorporating the recommendations of Tomorrow's Doctors. This paper investigates whether the PRHO (pre-registration house officer) educational supervisors who supervise Liverpool graduates believe that the reformed curriculum is producing house officers who have the competencies outlined by the General Medical Council. METHODS: Forty-one interviews were arranged with a sample of educational supervisors in the Mersey Deanery area to ascertain their views on the competencies of Liverpool PRHOs. The interviews were tape-recorded, transcribed and analysed. RESULTS: The supervisors felt that the PRHOs had been well prepared to work as PRHOs, and compared with traditional graduates they were actually better prepared for the job of PRHO. They saw the PRHOs as competent communicators with improved clinical skills, who had good attitudes, were aware of limitations, were team workers, good at history-taking and examination and had different approaches to finding information. There was a very confusing picture regarding basic knowledge, with some consultants expressing concerns, despite saying they had adequate knowledge to work as PRHOs. It emerged that there was little consensus about the knowledge base of the traditional or PBL graduates or what knowledge level was needed at this stage. DISCUSSION: Educational supervisors believe that the reformed curriculum in Liverpool is producing competent PRHOs who have been better prepared for the role of PRHO than previous graduates. It could be that uncertainty over knowledge base may be the price paid for reducing the factual burden and improving preparedness for professional practice.     

Implementation of problem-based learning in Asian medical schools and students' perceptions of their experience

BACKGROUND: Since the introduction of problem-based learning (PBL) at McMaster University in 1969, many medical schools in the USA, Canada and Europe have included PBL in their curricula. In the past decade, many medical schools in Asia have also done so. However, so far no one has questioned whether the outcomes expected of the learner in a PBL setting are applicable to students from different cultural upbringings. AIM: The aim of this study was to investigate the implementation of PBL in Asian medical schools, their students' perceptions of this new mode of teaching/learning and how the problems that have arisen may be overcome. METHOD: Published reports and conference presentations were gathered on the implementation of PBL in some Asian medical schools and comparisons of the experiences in PBL of Asian and students of other ethnic backgrounds. RESULTS: Most Asian medical schools and their students appear to be positive about adapting to PBL in their curriculum. The positive and negative observations appear to be similar to those experienced in non-Asian medical schools. The problems that arose for students in Asian medical schools in the early stages of implementing PBL appear to have been overcome after a period of adjustment with the help of supportive and open-minded tutors. The reports also suggested that trigger problems should be carefully designed to make them relevant and interesting for the students. CONCLUSION: Strong support from the academic administrators (dean and other staff responsible for implementation of the curriculum) in the introduction of PBL into the curriculum and careful training of both faculty and students appear to be key factors to ensure the successful implementation of PBL in Asian medical schools.     

Graduate entry to medical school? Testing some assumptions

BACKGROUND: Debate abounds regarding the most appropriate candidates to admit to medical school. This paper examines whether there is any advantage to admitting 'graduate' entrants over secondary school leavers on selected medical school and practice outcomes. AIM: To compare the medical school experiences, research and academic achievements and practice outcomes of graduates who entered 1 medical school in Australia directly from high school (secondary school entry) to those of graduates who entered with tertiary level education (tertiary entry). DESIGN AND METHODS: Cross-sectional study using a mail-out survey to graduates from the first 16 graduating years (1983-98 inclusive) of the University of Newcastle Medical School. RESULTS: Secondary school entrants were, on average, 8 years younger than tertiary entrants and were less likely to have received rural-based schooling. However, there were no differences with respect to gender or type of secondary school attended (public or private). Motivations for studying medicine did not generally differ according to entry type, except that more secondary students were motivated by parental expectations and more tertiary entrants were motivated by the need for professional independence and the desire to prevent disease. A greater proportion of tertiary entrants experienced stress at medical school. However, secondary students experienced more stress due to doubts about being a doctor, while tertiary entrants experienced more stress due to lack of leisure time, finances and balancing commitments. There were no significant differences between the groups in terms of academic performance (as measured by the award of medical school honours) or research outcomes (as measured by completion of a research degree during or after medical school training, publication of scientific papers or holding career posts in the research sciences). There were no differences in career positions held by clinicians, choice of general practice or another specialty as a career, practice location (rural or urban) or employment sector (public or private). CONCLUSION: There is no clear advantage, at least on the outcomes measured in this study, to limiting medical school entry to either those candidates from secondary school or those with tertiary backgrounds. Medical schools could reasonably broaden their selection criteria to include more graduate entry candidates in addition to secondary school leavers without compromising medical school and practice outcomes.     

Medical graduates evaluate the effectiveness of their education

Summary. In response to the General Medical Council's 1991 Consultative Document proposing changes to medical curricula, the King's College School of Medicine and Dentistry, London, established a steering group to plan a new curriculum. As part of this process a postal survey was undertaken of five cohorts of King's graduates to ascertain how well the medical course they had undertaken had equipped them for practice. The questionnaire explored the level of factual content, the adequacy of their skills training, and the development of personal attributes during the course. A total of 371 graduates replied, a response rate of 78%. Over 70% indicated that their education had satisfactorily equipped them for their medical practice. Significant differences were found, however, between those now practising in primary care and those in hospital medicine regarding the importance attached to different subjects within the curriculum, and also with respect to the personal attributes the graduates felt they had acquired. Both groups identified deficiencies in virtually all aspects of their skills training: clinical, analytical, communication, management and technical.
This feedback from some of the 'consumers' of medical education is now being used to assist the planning of the new King's curriculum.     

Target-focused medical emergency team training using a human patient simulator: effects on behaviour and attitude

CONTEXT: Full-scale simulation training is an accepted learning method for gaining behavioural skills in team-centred domains such as aviation, the nuclear power industry and, recently, medicine. In this study we evaluated the effects of a simulator team training method based on targets and known principles in cognitive psychology. METHODS: This method was developed and adapted for a medical emergency team. In particular, we created a trauma team course for novices, and allowed 15 students to practise team skills in 5 full-scale scenarios. Students' team behaviour was video-recorded and students' attitude towards safe teamwork was assessed using a questionnaire before and after team practice. RESULTS: Nine of 10 observed team skills improved significantly in response to practice, in parallel with a global rating of team skills. In contrast, no change in attitude toward safe teamwork was registered. CONCLUSION: The use of team skills in 5 scenarios in a full-scale patient simulator environment implementing a training method based on targets and known principles in cognitive psychology improved individual team skills but had no immediate effect on attitude toward safe patient care.     

Progression of junior doctors into higher specialist training

OBJECTIVES: To report on the transition of junior doctors into higher specialist training in the UK, following the Calman reforms and recent initiatives to promote training in general practice. Design and setting Postal questionnaire survey carried out in the UK. PARTICIPANTS: All graduates of 1996 from UK medical schools were surveyed in 2002. MAIN OUTCOME MEASURES: Outcome measures were considered to be details of applications, outcomes of applications, intentions to apply for specialist training, and career plans with regard to such issues as flexible training and work. RESULTS: Of 2312 responders, 39% had applied for specialist registrar (SpR) training and 35% for general practice (GP) training. Of applicants for SpR training, 68% were successful; 24% failed, almost all of whom intended to reapply, many after gaining research experience; 4% awaited a decision, and 4% had had another outcome (e.g. they withdrew their application). A sixth of responders intended to apply but had not yet done so. Of applicants for GP training, 95% had been successful. A further 2% intended to apply but had not yet done so. Responders viewed flexible and part-time training and work opportunities, and information about available posts, as being more widely available in general practice than in hospital practice. Half of the responders did not agree that their postgraduate training had been of a high standard. CONCLUSIONS: Progression into GP training seemed to have been a smoother and less protracted process than that into SpR training. Delayed applications for SpR training were common, and many of those who had applied had not been accepted. The use of research experience to strengthen a re-application was common.     

Predicting doctor performance outcomes of curriculum interventions: problem-based learning and continuing competence

Context  Problem-based learning (PBL) is an educational strategy designed to enhance self-assessment, self-directed learning and lifelong learning. The present study examines a peer review programme to determine whether the impact of PBL on continuing competence can be detected in practice.
Objectives  This study aimed to establish whether McMaster graduates who graduated between 1972 and 1991 were any less likely to be identified as having issues of competence by a systematic peer review programme than graduates of other Ontario medical schools.
Methods  We identified a total of 1166 doctors who had graduated after 1972 and had completed a mandated peer review programme. Of these, 108 had graduated from McMaster and 857 from other Canadian schools. School of graduation was cross-tabulated against peer rating. A secondary analysis examined predictors of ratings using multiple regression.
Results  We found that 4% of McMaster graduates and 5% of other graduates were deemed to demonstrate cause for concern or serious concern, and that 24% of McMaster doctors and 28% of other doctors were rated as excellent. These differences were not significant. Multiple regression indicated that certification by family medicine or a specialty, female gender and younger age were all predictors of practice outcomes, but school of graduation was not.
Conclusions  There is no evidence from this study that PBL graduates are better able to maintain competence than graduates of conventional schools. The study highlights potential problems in attempting to link undergraduate educational interventions to doctor performance outcomes.     

Does a new undergraduate curriculum based on Tomorrow's Doctors prepare house officers better for their first post? A qualitative study of the views of pre‐registration house officers using critical incidents

INTRODUCTION: In 1994 Manchester University introduced an integrated undergraduate medical course using problem-based learning (PBL) throughout. The study reported here explored whether there were any differences between the new course graduates (NCGs) and the traditional course graduates (TCGs) in the types of scenarios they recalled as 'critical incidents', or challenging cases, while working as pre-registration house officers (PRHOs). The focus is on differences rather than causal links. METHOD: We used semistructured interviews to generate our data. Twenty-four traditional course graduates and 23 new course graduates were interviewed approximately 3 months after starting their first PRHO placement. RESULTS: We identified 4 types of critical incidents relating to: clinical practice; limitations of competence; emotional involvement; and communication. Traditional course graduates reported difficulties in making patient management decisions, whereas the NCGs were better at dealing with uncertainty, knowing their limits and asserting their rights for support. Communication difficulties and coping with emotional involvement were common across both groups of graduates and hence remain problems in relation to being prepared for the role of a PRHO. CONCLUSIONS: Graduates of the new, integrated curriculum seemed to be much better at dealing with uncertainty, knowing their personal limits and asserting their rights for support when they felt these limits had been reached. Communication difficulties and emotional involvement remain major factors in the transition from student to PRHO.     

Predicting communication skills with a paper-and-pencil test

AIM: This study was conducted to investigate the value of a written knowledge test of communication skills for predicting scores on a performance test of communication skills. METHOD: A paper-and-pencil test of knowledge about communication skills and a performance test of communication skills, consisting of four stations with standardised patients, were administered to students of two classes of the medical schools of Maastricht and Leiden, the Netherlands. The results on these tests were compared. RESULTS: From the results of both instruments, the classes of the participating students could be recognised equally well: 60% correct qualifications of the classes by the knowledge test and 64% by the multiple station examination. Between the two tests an overall, disattenuated correlation of 0.60 was found (N=133, P < 0.01), suggesting moderate predictive value of the knowledge test for the performance test of communication skills. The correlation is stronger for students from Maastricht medical school than for their colleagues in Leiden. Correlation between the knowledge of communication skills test and other available test results of the participating Maastricht students is close to zero, suggesting that the test measures a distinct quality of students' competence. DISCUSSION: The paper-and-pencil test of knowledge of communication skills has predictive value for the performance of these skills, but this value seems to be less pronounced than similar findings for clinical procedural skills. The stronger relationship between 'knowing how' and 'showing' in the Maastricht student group might be indicative of an effect of the training format.     

Effectiveness of basic clinical skills training programmes: a cross-sectional comparison of four medical schools

OBJECTIVE: Training in physical diagnostic skills is an important part of undergraduate medical education. The objective of this study was to study the outcome of skills training at four medical schools. CONTEXT: At the time of the study, three schools had a traditional lecture-based curriculum and one school had a problem-based learning curriculum with a longitudinal skills training programme. All schools offer extended exposure to clerkships. METHOD: A cross-sectional study in four medical schools was performed, using a written test of skills that has good correlation with actual student performance. The scores attained from four student groups were compared within and between the four medical schools. A total of 859 volunteer students from the later four years at each medical school participated in the study. RESULTS: The mean scores in the traditional medical schools increased with the start of skill training and the hands-on experience offered during the clerkships. Students from the school with the longitudinal skills training programme and the problem-based learning approach had significantly higher mean scores at the start of the clerkships, and maintained their lead in the subsequent clinical years. CONCLUSIONS: Longitudinal skills training seems to offer the students a superior preparation for clerkships as well as influencing the students' learning abilities during the clerkships. The effect of the problem-based learning approach, also related to the innovative philosophy of the curriculum, could not be accounted for.     

Simulation in undergraduate medical education: bridging the gap between theory and practice

OBJECTIVE: To evaluate the use of simulation-based teaching in the medical undergraduate curriculum in the context of management of medical emergencies, using a medium fidelity simulator. DESIGN: Small groups of medical students attended a simulation workshop on management of medical emergencies. The workshop was evaluated in a post-course questionnaire. SUBJECTS: All Year 4 medical students allocated to the resuscitation rotation during the first half of 2002. MAIN OUTCOME MEASURES: Student perceptions of learning outcomes, the value of the simulation in the undergraduate curriculum and their self-assessed improved mastery of workshop material. RESULTS: A total of 33 students attended the workshop and all completed questionnaires. Students rated the workshop highly and found it a valuable learning experience. In all, 21 (64%) students identified teamwork skills as key learning points; 11 (33%) felt they had learnt how to approach a problem better, particularly in terms of using a systematic approach, and 12 (36%) felt they had learnt how to apply their theoretical knowledge in a clinical setting better. All 33 students were positive about the use of simulation in their training; 14 students wrote that simulation should be used more or should be mandatory in training; 5 students commented positively on the realism of the learning experience and a further 5 said they valued the opportunity to learn new skills in a safe environment. CONCLUSION: This study demonstrates that medical students value simulation-based learning highly. In particular, they value the opportunity to apply their theoretical knowledge in a safe and realistic setting, to develop teamwork skills and to develop a systematic approach to a problem. A medium fidelity simulator is a valuable educational tool in medical undergraduate education.     

The current state of medical education in Japan: a system under reform

CONTEXT: Not since just after World War II has there been as dramatic a change in the system of medical education in Japan as in the last several years. Medical school curricula are including more education that mimics clinical practice through problem-based learning, organ-based curricula and implementation of the objective structured clinical examination (OSCE). In response to criticism and concerns, the Japanese government has also implemented 2 major changes in the system of postgraduate medical education. First, a 2-year structured internship has been required of all medical school graduates; the first cohort to undertake this completed it in April 2006. Second, an internship matching system was adopted and first implemented in 2003. DISCUSSION: These reforms are leading to significant shifts in clinical education in Japan. Increasing numbers of medical graduates are entering residency programmes outside specialised university hospitals and core rotations place an increased emphasis on primary care. CONCLUSIONS: These changes in the training of young doctors suggest that the general clinical competency of doctors in Japan will improve in the coming years.