Influence of new educational technology on problem-based learning at Harvard Medical School

PURPOSE: Computers with 50-inch, wall-mounted plasma screens and broadband Internet access were installed in all small group tutorial rooms at Harvard Medical School. This study examines how the introduction of this educational technology impacted on the problem-based learning tutorials. METHOD: A total of 37 tutorial groups, stratified by year of student, were observed at separate timepoints (autumn 2002, spring 2003) to document the patterns of use of the technologies. Based on these observations, end-of-course surveys were developed and distributed to students and tutors. Observational field notes and open-ended survey responses were qualitatively analysed for themes. RESULTS: Using a 5-point rating scale, both students and tutors indicated that the technologies had a positive impact on their tutorials. In autumn 2002, plasma screens were used for an average of 17.8 and 22.1 minutes per 1-hour observation in Year 1 and 2 tutorials, respectively; in spring 2003, usage declined to 6.9 and 5.9 minutes, respectively. Resources utilised included Internet sites (54% total use time), PowerPoint presentations by students (22%), and course-specific postings (24%). Marked course-specific variation in usage was noted. Observational and survey data revealed that the technologies interrupted the flow of tutorial discussion. Students and tutors expressed concerns that the plasma screens might be altering the process of problem solving in the tutorials. CONCLUSIONS: Both students and tutors reported that the introduction of computers and wall-mounted plasma screens had impacted positively on tutorials. Questions were raised as to how this technology might alter tutorial dynamics. Further research will be needed to investigate these pedagogical concerns.     

Changes in student approaches to learning with the introduction of computer-supported problem-based learning

OBJECTIVES: To study changes in student approaches to learning following the introduction of computer-supported, problem-based learning. SETTING: Medical students at the University of Oslo undertake a 12-week period of clinical placement during their 10th term. In this period they continue to undertake problem-based learning (PBL) in the form of distributed problem-based learning (DPBL) in a computer-supported learning environment. DESIGN: A questionnaire focusing on learning styles, PBL, and information and communication technology (ICT) was distributed before and after the DPBL period. SUBJECTS: All students in their 10th term at the University of Oslo (n = 61). RESULTS: The introduction of DPBL did not seem to affect the participants' use of regulating strategies or their mental models of learning. After the DPBL period, group discussion and tutor input were reported to have less influence on students' self-study, while the students perceived themselves as being less active in groups and as expecting less from tutors. There was a relationship between perceived tutor influence and students' familiarity with ICT. The DPBL period seemed to increase students' task-related web accesses and use of experts, and to decrease their task-related use of textbooks and discussions with students outside the group. CONCLUSIONS: Students' general approaches to learning were not affected by the introduction of DPBL. However, there was a decrease in students' expectations concerning activity in the group and the importance of the tutor. These changes were related to students' familiarity with the use of computers. Web-based resources and experts became more important resources to the students during the DPBL period.     

Longterm effects of problem-based learning: a comparison of competencies acquired by graduates of a problem-based and a conventional medical school

BACKGROUND: Problem-based learning (PBL) as an approach to the instruction of medical students has attracted much attention in recent years. However, its effect on the performance of its graduates is the subject of considerable debate. This article presents data from a large-scale study among graduates of a problem-based medical school and those of a conventional medical school to contribute to this discussion. PURPOSE: To study the longterm effects of problem-based medical training on the professional competencies of graduates. METHODS: A questionnaire was sent to all graduates since 1980 of a problem-based and a conventional medical school. Participants were requested to rate themselves on 18 professional competencies derived from the literature. RESULTS: The graduates of the PBL school scored higher on 14 of 18 professional competencies. Graduates of the problem-based school rated themselves as having much better interpersonal skills, better competencies in problem solving, self-directed learning and information gathering, and somewhat better task-supporting skills, such as the ability to work and plan efficiently. There were no sizeable differences with regard to general academic competencies, such as conducting research or writing a paper. Graduates from the conventional school rated themselves as having slightly more medical knowledge. The findings were shown to be valid and robust against possible response bias. CONCLUSION: The findings suggest that PBL not only affects the typical PBL-related competencies in the interpersonal and cognitive domains, but also the more general work-related skills that are deemed important for success in professional practice.     

The influence of tutoring competencies on problems, group functioning and student achievement in problem-based learning

CONTEXT: Prominent factors in problem-based learning (PBL) are the problems to be solved, tutorial group functioning and tutors' competencies. These factors mutually affect one another and largely determine whether a powerful learning environment will be created. It is a tutor's task to stimulate active, self-directed, contextual and collaborative learning and display interpersonal behaviour that is conducive to students' learning. We investigated the effects of tutors' competencies on students' learning and on other variables, such as group functioning and student achievement. OBJECTIVES: We investigated whether tutors who stimulate active, self-directed, contextual and collaborative learning make better use of problems and meaningful contexts in PBL and also enhance group functioning. We also investigated whether the quality of problems has a positive impact on group functioning and whether group functioning advanced student achievements. METHODS: Questionnaires were used to collect data from students at the end of 11 modules in Years 1 and 2 of a PBL undergraduate medical curriculum. We used structural equation modelling to test the fit of a theoretical model representing the factors of interest and their relationships. RESULTS: Stimulation of active and constructive learning, self-directed learning and collaborative learning by tutors enhanced the quality of the problems and group functioning. The quality of the problems promoted group functioning, which was found to have a positive effect on student achievement. CONCLUSIONS: Tutors' competencies had a positive effect on the learning of students. This suggests that it would be worthwhile including these competencies in staff development.     

Clinical teachers and problem-based learning: a phenomenological study

AIM: To explore how clinicians perceive their roles in problem-based medical education, and how closely those perceptions link to the curriculum they teach. METHOD: All 14 general physicians in a teaching hospital took part in 6 semistructured discussions, which were analysed phenomenologically. RESULTS: Third year clinical teaching was described in terms that bore little relation to problem-based learning (PBL). Teachers placed great importance on the social dimension of professional learning. They expressed strongly positive affects towards learners and their learning that they found hard to express as PBL tutors. Their narratives of education were remarkably divorced from modern day clinical practice. CONCLUSIONS: Problem-based method lacked some important conditions for professional teaching and learning. Traditional apprenticeship is unsustainable under present day conditions of practice. There is a need for new educational methods that help the learner to build a professional identity through social interaction with practitioners.     

The effectiveness of problem-based learning compared to traditional teaching in undergraduate psychiatry

OBJECTIVES: A change from traditional to problem-based learning (PBL) methods in a psychiatry attachment was evaluated by comparing the learning styles, attitudes to psychiatry and examination performance of 2 cohorts of students. It was hypothesised that the PBL curriculum would result in increased deep learning, decreased surface learning, more favourable attitudes to psychiatry and improved examination performance. It was predicted that students' examination success would be related to the use of deep and strategic learning and favourable attitudes. METHODS: Consecutive cohorts of Year 2 clinical students taught using a traditional psychiatry curriculum (n = 188) and a PBL curriculum (n = 191) were compared. Students completed the Study Process Questionnaire to assess their learning styles and the Attitudes to Psychiatry Scale at the beginning and end of the attachment. Students completed 2 end-of-attachment examinations, a multiple-choice paper and a viva. RESULTS: The PBL curriculum resulted in significantly better examination performance than did the traditional teaching curriculum, both for multiple-choice questions and the viva. No differences in learning styles or attitudes to psychiatry were found between the curricula. Students were significantly more successful in the examinations if they had received the PBL curriculum, were female, and used strategic learning. CONCLUSIONS: Examination performance indicated that the PBL curriculum was more successful than the previous course, but that this improvement was not due to students using more effective learning styles or having more favourable attitudes towards psychiatry. It is possible that students learned more effectively during the teaching sessions in the PBL curriculum, but did not change their preferred learning styles.     

Supporting the problem-based learning process in the clinical years: evaluation of an online Clinical Reasoning Guide

PURPOSE: Implementing problem-based learning (PBL) in the clinical years of a medical degree presents particular challenges. This study investigated the effectiveness of using an online Clinical Reasoning Guide to assist integration of PBL in the clinical setting and promote further development of students' clinical reasoning abilities. METHOD: A total of 52 students in 6 PBL groups, together with their 6 clinical tutors, participated in the study. Data were analysed from videotaped observations of tutorial activity and follow-up, semistructured interviews. RESULTS: From both the student facilitators' and the clinical tutors' perspectives, the Guide proved an effective tool for augmenting the PBL process in clinical settings and promoting the development of clinical reasoning. By combining computer-aided learning with collaborative PBL tutorials it promoted individual as well as collaborative reasoning. There is also evidence to suggest that the Guide prompted students to look more critically at their own, their colleagues' and other clinicians' reasoning processes.     

Growth of knowledge in psychiatry and behavioural sciences in a problem-based learning curriculum

PURPOSE: To evaluate the effectiveness of undergraduate medical education in the domains of psychiatry and behavioural sciences, we examined the growth of knowledge in those disciplines in a 6-year, problem-based learning (PBL) curriculum. Psychiatry and behavioural sciences are taught in the 4 preclinical years and in the psychiatric clerkship. The integrative nature of this PBL curriculum led us to hypothesise that the knowledge growth curves for these disciplines are similar and show a steady upward trend throughout the curriculum. METHODS: All items pertaining to psychiatry and behavioural sciences in the progress tests administered in the period from September 1993 through May 2001 were identified. For those items, the percentage of correct scores in the 6 year groups were considered a multivariate observation reflecting knowledge growth across the 6-year programme. RESULTS: Knowledge growth for psychiatry and behavioural sciences increased significantly, from 12% to 59% and from 28% to 60%, respectively, between Year 1 and the end of Year 6. Apparently, students know more about behavioural sciences than about psychiatry when they enter medical school, but this difference vanishes in the last 2 years of training. Moreover, the growth curves for psychiatry and behavioural sciences started to level off after Years 3 and 4, respectively, with no additional significant growth in any of the later years. CONCLUSIONS: Psychiatry and behavioural sciences showed different patterns of knowledge growth and the 2 growth curves levelled off in Years 5 through 6. Because a student-centred, horizontally and vertically integrated PBL curriculum is aimed at effecting steady growth in knowledge in all disciplines, the slowdown in growth in the later years was among the reasons for initiating a major curricular innovation in 2001.     

Towards valid measures of self-directed clinical learning

AIM: To compare the validity of different measures of self-directed clinical learning. METHODS: We used a quasi-experimental study design. The measures were: (1) a 23-item quantitative instrument measuring satisfaction with the learning process and environment; (2) free text responses to 2 open questions about the quality of students' learning experiences; (3) a quantitative, self-report measure of real patient learning, and (4) objective structured clinical examination (OSCE) and progress test results. Thirty-three students attached to a single firm during 1 curriculum year in Phase 2 of a problem-based medical curriculum formed an experimental group. Thirty-one students attached to the same firm in the previous year served as historical controls and 33 students attached to other firms within the same module served as contemporary controls. After the historical control period, experimental group students were exposed to a complex curriculum intervention that set out to maximise appropriate real patient learning through increased use of the outpatient setting, briefing and supported, reflective debriefing. RESULTS: The quantitative satisfaction instrument was insensitive to the intervention. In contrast, the qualitative measure recorded a significantly increased number of positive statements about the appropriateness of real patient learning. Moreover, the quantitative self-report measure of real patient learning found high levels of appropriate learning activity. Regarding outpatient learning, the qualitative and quantitative real patient learning instruments were again concordant and changed in the expected direction, whereas the satisfaction measure did not. An incidental finding was that, despite all attempts to achieve horizontal integration through simultaneously providing community attachments and opening up the hospital for self-directed clinical learning, real patient learning was strongly bounded by the specialty interest of the hospital firm to which students were attached. Assessment results did not correlate with real patient learning. CONCLUSIONS: Both free text responses and students' quantitative self-reports of real patient learning were more valid than a satisfaction instrument. One explanation is that students had no benchmark against which to rate their satisfaction and curriculum change altered their tacit benchmarks. Perhaps the stronger emphasis on self-directed learning demanded more of students and dissatisfied those who were less self-directed. Results of objective, standardised assessments were not sensitive to the level of self-directed, real patient learning. Despite an integrated curriculum design that set out to override disciplinary boundaries, students' learning remained strongly influenced by the specialty of their hospital firm.     

The patient as text: a challenge for problem-based learning

OBJECTIVES: To explore the values and assumptions underlying problem-based learning (PBL) cases through narrative analysis, in order to consider the ways by which paper cases may affect student attitudes and values. METHODS: Randomly chosen PBL cases from the first year curriculum at Dalhousie University medical school (n = 10) were coded by 3 independent reviewers attending to narrative components. RESULTS: The cases generally used spare, objective language, used the passive voice, eliminated agency, and employed linguistic markers to encode scepticism about patient reports. There was almost no sense of the presence of the patient as person in these cases in terms of their words, feelings, or their social and cultural context. The almost complete exclusion of the preferences and priorities of the patient was striking. CONCLUSION: The sample is small, the results only suggestive. Yet it appears that the cases used in PBL may unnecessarily, even unintentionally, encourage student detachment from the messiness of real patients' lives and emotions. Positioning a particular way of seeing - the doctor's gaze - as normative renders less visible the choices that are being made whenever an account is constructed. Including multiple voices in a case would complicate that tidy reduction of choices. Ongoing attempts to enrich the case format should be encouraged. At the same time, students may benefit from being taught the skills for critical analysis of the case itself.     

Implementation of problem-based learning in Asian medical schools and students' perceptions of their experience

BACKGROUND: Since the introduction of problem-based learning (PBL) at McMaster University in 1969, many medical schools in the USA, Canada and Europe have included PBL in their curricula. In the past decade, many medical schools in Asia have also done so. However, so far no one has questioned whether the outcomes expected of the learner in a PBL setting are applicable to students from different cultural upbringings. AIM: The aim of this study was to investigate the implementation of PBL in Asian medical schools, their students' perceptions of this new mode of teaching/learning and how the problems that have arisen may be overcome. METHOD: Published reports and conference presentations were gathered on the implementation of PBL in some Asian medical schools and comparisons of the experiences in PBL of Asian and students of other ethnic backgrounds. RESULTS: Most Asian medical schools and their students appear to be positive about adapting to PBL in their curriculum. The positive and negative observations appear to be similar to those experienced in non-Asian medical schools. The problems that arose for students in Asian medical schools in the early stages of implementing PBL appear to have been overcome after a period of adjustment with the help of supportive and open-minded tutors. The reports also suggested that trigger problems should be carefully designed to make them relevant and interesting for the students. CONCLUSION: Strong support from the academic administrators (dean and other staff responsible for implementation of the curriculum) in the introduction of PBL into the curriculum and careful training of both faculty and students appear to be key factors to ensure the successful implementation of PBL in Asian medical schools.     

Predicting doctor performance outcomes of curriculum interventions: problem-based learning and continuing competence

Context  Problem-based learning (PBL) is an educational strategy designed to enhance self-assessment, self-directed learning and lifelong learning. The present study examines a peer review programme to determine whether the impact of PBL on continuing competence can be detected in practice.
Objectives  This study aimed to establish whether McMaster graduates who graduated between 1972 and 1991 were any less likely to be identified as having issues of competence by a systematic peer review programme than graduates of other Ontario medical schools.
Methods  We identified a total of 1166 doctors who had graduated after 1972 and had completed a mandated peer review programme. Of these, 108 had graduated from McMaster and 857 from other Canadian schools. School of graduation was cross-tabulated against peer rating. A secondary analysis examined predictors of ratings using multiple regression.
Results  We found that 4% of McMaster graduates and 5% of other graduates were deemed to demonstrate cause for concern or serious concern, and that 24% of McMaster doctors and 28% of other doctors were rated as excellent. These differences were not significant. Multiple regression indicated that certification by family medicine or a specialty, female gender and younger age were all predictors of practice outcomes, but school of graduation was not.
Conclusions  There is no evidence from this study that PBL graduates are better able to maintain competence than graduates of conventional schools. The study highlights potential problems in attempting to link undergraduate educational interventions to doctor performance outcomes.     

Development of a teaching style inventory for tutor evaluation in problem-based learning

OBJECTIVE: To develop and validate a self-rating instrument to assess teaching styles among tutors in problem-based learning (PBL). METHODS: The development of the teaching style inventory (TSI) was based theoretically on four types of teaching behaviours: the assertive, suggestive, collaborative and facilitative styles, as proposed by Bibace et al. A 35-item questionnaire was generated and evaluated for content validity by a group of experienced tutors. The questionnaire was mailed to 196 tutors at the National Taiwan University College of Medicine. The results were submitted for item analysis, internal consistency testing and exploratory factor analysis. Longterm test-retest reliability was assessed by a sample of 50 tutors after a 6-month interval. RESULTS: Finally, 118 tutors returned the questionnaires. In the item reduction process, seven items were excluded due to low interscale correlation. Principle component factoring yielded a three-factor solution that accounted for 48.5% of the total variance. Internal consistency coefficients of the four hypothetical domains ranged from 0.73 to 0.83. All domains correlated to each other as expected. Assertive and facilitative styles, which are theoretically opposite teaching styles, showed a negative correlation with each other. Most of the items of each hypothetical domain correlated better with their own domain than with other domains. Longterm test-retest correlations of the four domains ranged from 0.54 to 0.81. CONCLUSION: The TSI demonstrated high internal consistency reliability, acceptable longterm test-retest reliability, and construct validity. Further psychometric testing should focus on applicability to other populations, predictive validity and short-term test-retest reliability. This instrument can be used by programme directors for the recruitment of tutors and can also be used to increase the self-awareness of tutors.     

Simulation in undergraduate medical education: bridging the gap between theory and practice

OBJECTIVE: To evaluate the use of simulation-based teaching in the medical undergraduate curriculum in the context of management of medical emergencies, using a medium fidelity simulator. DESIGN: Small groups of medical students attended a simulation workshop on management of medical emergencies. The workshop was evaluated in a post-course questionnaire. SUBJECTS: All Year 4 medical students allocated to the resuscitation rotation during the first half of 2002. MAIN OUTCOME MEASURES: Student perceptions of learning outcomes, the value of the simulation in the undergraduate curriculum and their self-assessed improved mastery of workshop material. RESULTS: A total of 33 students attended the workshop and all completed questionnaires. Students rated the workshop highly and found it a valuable learning experience. In all, 21 (64%) students identified teamwork skills as key learning points; 11 (33%) felt they had learnt how to approach a problem better, particularly in terms of using a systematic approach, and 12 (36%) felt they had learnt how to apply their theoretical knowledge in a clinical setting better. All 33 students were positive about the use of simulation in their training; 14 students wrote that simulation should be used more or should be mandatory in training; 5 students commented positively on the realism of the learning experience and a further 5 said they valued the opportunity to learn new skills in a safe environment. CONCLUSION: This study demonstrates that medical students value simulation-based learning highly. In particular, they value the opportunity to apply their theoretical knowledge in a safe and realistic setting, to develop teamwork skills and to develop a systematic approach to a problem. A medium fidelity simulator is a valuable educational tool in medical undergraduate education.     

On the relationship between group functioning and study success in problem-based learning

INTRODUCTION: In problem-based learning (PBL), discussion in the tutorial group plays a central role in stimulating student learning. Problems are the principal input for stimulating discussion. The quality of discussion is assumed to influence student learning and, in the end, study success. AIMS: To investigate the relationships between aspects of group functioning and study success. METHODS: First-year medical students (n = 116), forming 12 PBL groups, completed a 21-item questionnaire on various aspects of a PBL session. At the end of the unit, a course examination was administered. Scales were constructed and reliability analyses conducted. RESULTS: Group functioning and case quality were strongly correlated with students' grades in a course examination. Further, students' perceptions of group functioning, case quality and the quality of their own contribution were linked strongly with each other. CONCLUSIONS: Group functioning, case quality and study success are associated with each other in PBL. The interaction between these aspects of PBL in promoting learning calls for further investigation.     

A web-based presentation of an undergraduate clinical skills curriculum

AIMS: The aim of this study was to use information and communications technology to present a curriculum of clinical skills in a user-friendly format. SETTING: A UK undergraduate medical school with a problem-based curriculum and a strong emphasis on proficiency in clinical skills. STUDY DESIGN: Case study describing the qualitative analysis of users' requirements and development of a web-based learning portfolio. EVALUATION: The study involved direct observation of users during a 'think-aloud' protocol, a validated software users' measurement inventory and a 17-item questionnaire designed to test whether 'SkillsBase' met its users' requirements. RESULTS: Students wanted a clear and flexible presentation of their skills curriculum that was easy to navigate, offered instructional material and standards for self- and peer assessment, offered useful Internet links, allowed them to compare their progress with school standards and peer norms, and could be used as a learning portfolio. During the think-aloud protocol, students made very few errors in data interpretation or navigation, and found SkillsBase easy to learn and aesthetically pleasing to use. They rated it higher on all measures of usability than standard commercial software. The questionnaire showed that it met most aspects of its design specification, although many students were doubtful that they would use its reflective function. It is available for inspection at http://www.skillsbase.man.ac.uk/. CONCLUSIONS: SkillsBase meets the design specification for a training and reflective aid to learning clinical skills and is very usable.     

Teaching and learning in the clinical setting: a qualitative study of the perceptions of students and teachers

Objective  To describe the perceptions of medical students and clinical teachers of teaching and learning in the clinical setting.
Design  Qualitative study of focus groups with undergraduate medical students and semistructured interviews with hospital consultant clinical teachers.
Setting  The School of Medicine, University of Leeds and the Leeds Teaching Hospitals Trust, UK.
Participants  Fourth year medical students and consultant clinical teachers.
Main outcome measures  Analysis of narratives to identify students' perceptions of clinical teaching and consultants' views of their delivery of undergraduate clinical teaching.
Results  Students believed in the importance of consultant teaching and saw consultants as role models. However, they perceived variability in the quality and reliability of teaching between physicians and surgeons. Some traditional teaching venues, especially theatre, are believed to be of little clinical importance. Generally, consultants enjoyed teaching but felt under severe pressure from other commitments. They taught in a range of settings and used various teaching strategies, not all of which were perceived to be 'teaching' by students.
Conclusions  While students and teachers are educational partners, they are not always in agreement about the quality, quantity, style or appropriate setting of clinical teaching. To enable teachers to provide more high quality teaching, there needs to be support, opportunities and incentives to understand curricular developments and acquire teaching skills.     

Measuring the effect of interprofessional problem-based learning on the attitudes of undergraduate health care students

OBJECTIVES: This study aimed to measure the improvement in attitudes towards interprofessional collaboration of undergraduate health care students who have a single module of interprofessional problem-based learning (PBL) using real patients as triggers integrated into their curricula. DESIGN: A dedicated module, consisting of 5 PBL seminars, was integrated into the undergraduate medical, nursing and physiotherapy curricula at the participating institutions. Seminar groups consisted of students from a single profession in the control group, and of evenly distributed students from the participating professions in the intervention group. The Interdisciplinary Education Perception Scale was used to measure improvements in attitudes towards interprofessional co-operation. Patients, faculty members and students were included in the evaluation of the interprofessional module and their comments examined for indications of adverse effects of the use of patients in this setting. RESULTS: A total of 177 students were recruited into the study and assigned to 1 of 16 seminar groups, all of which attended the complete module, 8 in control mode and 8 in intervention mode. Statistically significant improvements could be identified in the overall attitudes of male students in the intervention group, and in attitudes pertaining to the competence and autonomy of individuals in one's own profession in the intervention group as a whole. No significant improvements were detected in the control group. No adverse effects of the use of real patients came to light. CONCLUSIONS: The integration of an interprofessional educational module that requires limited student and faculty time in undergraduate health care curricula may be proven to have an effect. The Interdisciplinary Education Perception Scale may be suitable for measuring such effect. Real patients may continue to contribute to education in this setting.