The influence of tutoring competencies on problems, group functioning and student achievement in problem-based learning

CONTEXT: Prominent factors in problem-based learning (PBL) are the problems to be solved, tutorial group functioning and tutors' competencies. These factors mutually affect one another and largely determine whether a powerful learning environment will be created. It is a tutor's task to stimulate active, self-directed, contextual and collaborative learning and display interpersonal behaviour that is conducive to students' learning. We investigated the effects of tutors' competencies on students' learning and on other variables, such as group functioning and student achievement. OBJECTIVES: We investigated whether tutors who stimulate active, self-directed, contextual and collaborative learning make better use of problems and meaningful contexts in PBL and also enhance group functioning. We also investigated whether the quality of problems has a positive impact on group functioning and whether group functioning advanced student achievements. METHODS: Questionnaires were used to collect data from students at the end of 11 modules in Years 1 and 2 of a PBL undergraduate medical curriculum. We used structural equation modelling to test the fit of a theoretical model representing the factors of interest and their relationships. RESULTS: Stimulation of active and constructive learning, self-directed learning and collaborative learning by tutors enhanced the quality of the problems and group functioning. The quality of the problems promoted group functioning, which was found to have a positive effect on student achievement. CONCLUSIONS: Tutors' competencies had a positive effect on the learning of students. This suggests that it would be worthwhile including these competencies in staff development.     

Validation of a short questionnaire to assess the degree of complexity and structuredness of PBL problems

BACKGROUND: In problem-based learning (PBL), problems represent the starting point of students' learning activities. Therefore, the quality of these problems should be high, in that they should be of an adequate level of complexity and structuredness. Previous research has proposed several guidelines for constructing problems, but some of them are rather vague and are not based on empirical evidence. The present study aimed to validate a short questionnaire that can be used to assess the degree of complexity and structuredness of PBL problems. METHOD: This paper outlines Jonassen's theory, on which the questionnaire is based, and its relationship and applicability to PBL problems. The questionnaire was validated by means of confirmatory factor analysis. RESULTS: The results showed that students were able to distinguish PBL problems that were too simple and those that were too well-structured, but found it difficult to distinguish problems that were too complex or too ill-structured. CONCLUSION: The questionnaire may be used to measure the levels of complexity and structuredness of a problem as perceived by students and can provide teachers with feedback about the quality of problems.     

Current perspectives on medical education in China

CONTEXT: China has a long tradition of education and medicine. However, limited economic conditions and a huge population mean that further development of medical education in China must be tailored to meet the country's needs. OBJECTIVES: The aim of this paper is to describe current medical education practice in China with reference to the general and historical purposes of education in China and how they have affected and continue to affect student learning. Reference is also made to both Western medicine and traditional Chinese medicine. METHODS: It is argued that traditional educational practices in China have encouraged rote learning and that creativity is not cultivated. This affects the way many Chinese students learn medicine. Since 1949, the Chinese medical education system has developed according to its own needs. The current system for training medical students is complex, with medical school curricula lasting 3-8 years. However, medical education reform is taking place and new teaching methods are being introduced in some schools. DISCUSSION: Medical education is important to China's large population. The undergraduate medical education system is being streamlined and national standards are being established. Innovations in medical education have recently been encouraged and supported, including the adoption of problem-based learning. It is important that the momentum is kept up so that the health care of a fifth of the world's population is assured.     

Predicting doctor performance outcomes of curriculum interventions: problem-based learning and continuing competence

Context  Problem-based learning (PBL) is an educational strategy designed to enhance self-assessment, self-directed learning and lifelong learning. The present study examines a peer review programme to determine whether the impact of PBL on continuing competence can be detected in practice.
Objectives  This study aimed to establish whether McMaster graduates who graduated between 1972 and 1991 were any less likely to be identified as having issues of competence by a systematic peer review programme than graduates of other Ontario medical schools.
Methods  We identified a total of 1166 doctors who had graduated after 1972 and had completed a mandated peer review programme. Of these, 108 had graduated from McMaster and 857 from other Canadian schools. School of graduation was cross-tabulated against peer rating. A secondary analysis examined predictors of ratings using multiple regression.
Results  We found that 4% of McMaster graduates and 5% of other graduates were deemed to demonstrate cause for concern or serious concern, and that 24% of McMaster doctors and 28% of other doctors were rated as excellent. These differences were not significant. Multiple regression indicated that certification by family medicine or a specialty, female gender and younger age were all predictors of practice outcomes, but school of graduation was not.
Conclusions  There is no evidence from this study that PBL graduates are better able to maintain competence than graduates of conventional schools. The study highlights potential problems in attempting to link undergraduate educational interventions to doctor performance outcomes.     

Evaluation of self-directed clinical education: validation of an instrument

AIM: To explore the evaluation of self-directed, integrated clinical education. METHODS: We delivered a quantitative and qualitative, self-report questionnaire to students through their web-based learning management system. The questionnaire was distributed 4 times over 1 year, each time in 2 parts. A generic part evaluated boundary conditions for learning, teaching activities and "real patient learning". Factor analysis with varimax rotation was used to validate the constructs that made up the scale and to stimulate hypotheses about how they interrelated. A module-specific part evaluated real patient learning of the subject matter in the curriculum. RESULTS: A total of 101 students gave 380 of a possible 404 responses (94%). The generic data loaded onto 4 factors, corresponding to: firm quality; hospital-based teaching and learning; community and out-patient learning, and problem-based learning (PBL). A 5-item quality index had content, construct and criterion validity. Quality differed greatly between firms. Self-evaluation of module-specific, real patient learning was also valid. It was strongly influenced by the specialty interests of hospital firms. CONCLUSIONS: Quality is a multidimensional construct. Self-report evaluation of real patient learning is feasible, and could be capitalised on to promote reflective self-direction. The social and material context of learning is an important dimension of educational quality.     

Does problem‐based learning lead to deficiencies in basic science knowledge? An empirical case on anatomy

INTRODUCTION: Problem-based learning (PBL) is supposed to enhance the integration of basic and clinical sciences. In a non-integrative curriculum, these disciplines are generally taught in separate courses. Problem-based learning students perceive deficiencies in their knowledge of basic sciences, particularly in important areas such as anatomy. Outcome studies on PBL show controversial results, sometimes indicating that medical students at PBL schools have less knowledge of basic sciences than do their colleagues at more traditional medical schools. We aimed to identify differences between PBL and non-PBL students in perceived and actual levels of knowledge of anatomy. METHODS: Samples of Year 4 students in all eight medical schools in the Netherlands completed a questionnaire on perceived knowledge and took part in a computerised anatomy test consisting of both clinically contextualised items and items without context. RESULTS: Problem-based learning students were found to have the same perceived level of anatomy knowledge as students at other medical schools. Differences in actual levels of knowledge were found between schools. No significant effects on knowledge levels were found for PBL schools versus non-PBL schools. CONCLUSION: The results of this study show that PBL does not result in a lower level of anatomy knowledge than more traditional educational approaches. It remains to be ascertained whether the levels students attain are adequate. Subjects for further study are the desired level of anatomy knowledge at the end of undergraduate medical education and the effectiveness of basic science learning within a clinical context and with repetition over the course of the curriculum.     

Evaluating the impact of moving from discipline-based to integrated assessment

BACKGROUND: The move from discipline-based to problem-based learning (PBL) at Adelaide University in 2000 offered exciting opportunities to integrate the teaching and learning of the basic and clinical sciences for medical undergraduates. However, several cohorts of students still needed to progress through the first 3 years of the more traditional curriculum. Paradoxically, their readiness to function in the integrated learning and assessment environment of the last 3 years was assessed in 7 separate discipline-based examinations at the end of third year. When considerable examination-related stress was noted in the 1997 cohort and students petitioned formally for a reduced examination load, it was considered to be time for assessment to lead the way in integrating the disciplines. AIM: After introducing third year integrated written assessments in 1998, we aimed to develop an integrated practical examination (IPE) linking theory to practice, and evaluate its impact on staff and students. METHODS: After extensive staff collaboration, a structured objective multistation IPE was developed and administered in 1999 and 2000. Its utility was evaluated using a model proposed earlier. RESULTS: Assessment validity was maximised by an extensive item review process. Reliability, as measured by Cronbach's alpha, was 0.79 and 0.80 in 1999 and 2000, respectively. An independent evaluation yielded qualitative data on the examination's educational impact, cost and acceptability. CONCLUSIONS: Investing time in changing from discipline-based to integrated assessment, integrating theory and practice, resulted in gains in assessment reliability, validity and educational impact on both staff and students.     

Supporting the problem-based learning process in the clinical years: evaluation of an online Clinical Reasoning Guide

PURPOSE: Implementing problem-based learning (PBL) in the clinical years of a medical degree presents particular challenges. This study investigated the effectiveness of using an online Clinical Reasoning Guide to assist integration of PBL in the clinical setting and promote further development of students' clinical reasoning abilities. METHOD: A total of 52 students in 6 PBL groups, together with their 6 clinical tutors, participated in the study. Data were analysed from videotaped observations of tutorial activity and follow-up, semistructured interviews. RESULTS: From both the student facilitators' and the clinical tutors' perspectives, the Guide proved an effective tool for augmenting the PBL process in clinical settings and promoting the development of clinical reasoning. By combining computer-aided learning with collaborative PBL tutorials it promoted individual as well as collaborative reasoning. There is also evidence to suggest that the Guide prompted students to look more critically at their own, their colleagues' and other clinicians' reasoning processes.     

What do students actually do on an internal medicine clerkship? A log diary study

BACKGROUND: There are limited data on the amount of time students spend on teaching and learning while on internal medicine clerkships, and existing data suggest a wide international variation. Community-based teaching of internal medicine is now widespread; but its strengths and weaknesses compared to traditional hospital based teaching are still unclear. AIM: To determine the proportion of time students spend on different activities on an internal medicine clerkship, and to determine whether this differs in general practice and in hospital. In addition we aimed to determine students' views on the educational value and enjoyment of various activities. METHODS: Prospective completion of log diaries recording student activities. Each student was asked to complete the diary for two separate weeks of their internal medicine clerkship: one week of general practice-based teaching and one week of hospital-based teaching. RESULTS: The response rate was 68% (88/130). Students spent approximately 5.5 h per day on teaching and learning activities in both environments, with more time (50 min vs. 30 min, P = 0.007) on unsupervised interaction with patients in hospital than in general practice, and more time (53 min vs. 21 min, P < 0.001) undergoingassessment in general practice than in hospital. Standard deviations were wide, demonstrating the heterogeneous nature of the data. Students perceived supervised interaction with patients and teaching by doctors as the most educational activities in both environments, but found it even more educationally valuable and enjoyable in general practice than in hospital (mean score for educational value: 4.27 in general practice, 3.88 in hospital, P = 0.048; mean score for enjoyment 4.13 in general practice, 3.66 in hospital, P = 0.03). CONCLUSIONS: Students greatly value interactions with patients, perceiving these as both educational and enjoyable. Curriculum planners must continue to place patient-based learning at the centre of undergraduate medical education. The heterogeneity of the data suggests that individual students have very different experiences, despite apparently similar timetables.