Problem-solving skills, solving problems and problem-based learning

This paper reviews the empirical evidence in support of the three concepts in the title. To the extent that a skill should be a general strategy, applicable in a variety of situations, and independent of the specific knowledge of the situation, there is little evidence that problem-solving skills, as described and measured in medical education, possess these characteristics. Instead there is an accumulation of evidence that expert problem-solving in medicine is dependent on (I) a wealth of prior specific experiences which can be used in routine solution of problems by pattern recognition processes, and (2) elaborated conceptual knowledge applicable to the occasional problematic situation. The use of problem-based learning (PBL) as an educational strategy is explored. In particular, the evidence suggesting the compatibility of PBL with this theory of expertise is discussed. Finally, I review some issues in the design of PBL curricula from the perspective of the proposed model of expertise.     

Predicting doctor performance outcomes of curriculum interventions: problem-based learning and continuing competence

Context  Problem-based learning (PBL) is an educational strategy designed to enhance self-assessment, self-directed learning and lifelong learning. The present study examines a peer review programme to determine whether the impact of PBL on continuing competence can be detected in practice.
Objectives  This study aimed to establish whether McMaster graduates who graduated between 1972 and 1991 were any less likely to be identified as having issues of competence by a systematic peer review programme than graduates of other Ontario medical schools.
Methods  We identified a total of 1166 doctors who had graduated after 1972 and had completed a mandated peer review programme. Of these, 108 had graduated from McMaster and 857 from other Canadian schools. School of graduation was cross-tabulated against peer rating. A secondary analysis examined predictors of ratings using multiple regression.
Results  We found that 4% of McMaster graduates and 5% of other graduates were deemed to demonstrate cause for concern or serious concern, and that 24% of McMaster doctors and 28% of other doctors were rated as excellent. These differences were not significant. Multiple regression indicated that certification by family medicine or a specialty, female gender and younger age were all predictors of practice outcomes, but school of graduation was not.
Conclusions  There is no evidence from this study that PBL graduates are better able to maintain competence than graduates of conventional schools. The study highlights potential problems in attempting to link undergraduate educational interventions to doctor performance outcomes.     

The reliability of a hypothesis generation and testing task

The purpose of this paper is to present results of initial experience with a clinical reasoning task which assesses two clearly defined aspects of clinical problem solving. Fourteen senior and 40 junior medical students at the University of Michigan Medical School participated in this study. They were given three clinical reasoning problems--the hypothesis generation and testing tasks (HG & T). As suggested by the name, two specifically defined components of clinical problem-solving, developing the initial hypotheses or differential and then testing hypotheses, were evaluated by these tasks. The findings of this study indicate that hypothesis generation and testing can be reliably evaluated with between seven and ten tasks. The results of this study suggest that reliable assessments of specific components of clinical problem-solving can be developed.     

Tutoring in problem-based learning: a teacher development process

Summary: Summary . Undergraduate medical curricula have become increasingly innovative in order to better prepare their graduates to enter practice, with the most notable innovation being the introduction of problem-based learning (PBL). This paper describes Dalhousie University's transition to PBL, from a teacher development perspective.
The paper reinforces the need for a well-designed teacher development process that is carefully implemented and evaluated in order to ensure a successful curriculum change. A seven-stage process for teacher development of tutors at Dalhousie is described, and programme evaluation data are reported from both students' and teachers' points of view. The results from the evaluation are very positive and suggest ways of improving the teacher development process. These improvements are described, as well as future plans in this area.     

Integrating human factors into the medical curriculum

Background  The study of human factors is a scientific discipline that deals with the interactions between human beings and the elements of a system. This is important because shortcomings in these areas, if unchecked, can result in adverse outcomes. Research into human factors in industries where safety is paramount has provided the basis of countermeasures against human error. Adverse outcomes in medicine resulting from human error exact a high cost in both patient suffering and financial outlay.
CRM training  One of the approaches used to minimise the effect of human error is to train people in a set of knowledge, skills and attitudes that underpin the domain-specific competencies for that profession. These are referred to as non-technical skills (NTS). In aviation, such an approach has been shown to be both translatable from the training environment to the workplace and effective in reducing adverse outcomes.
Discussion  Medicine has incorporated this style of training, usually centred around simulator-based courses, but as yet in a piecemeal, episodic fashion which relies on participants volunteering to attend courses. Unlike other industries there is no systematic approach to linking the content of this teaching with the more conventional range of topics. As a consequence it is difficult to assess the impact of human factors training in medicine. This is partly because very little work has been done to date in identifying the key non-technical skills required in medicine, and the overall experience of workplace based assessment is limited. Lessons from other high reliability organisations may help to address the main challenges of developing the content, integrating it into the curriculum, reinforcing the concepts in the workplace through staff development and establishing its role in summative assessment.     

Performance evaluation of graduates from a community-based curriculum: the housemanship period at Gezira

The housemanship performance of the first two classes of the University of Gezira Medical School was assessed on 107 occasions by using a questionnaire administered to the senior doctors (consultants) in the units in which they worked and the data, which covered 32% of all housemanship rotations taken by these graduates, were analysed. On a 5-point scale (poor to excellent) the rating of performance for 26 questions was average and above (S = 3: + S = 4: + S = 5) in 95% of instances. The good and excellent rating (S = 4 + S = 5) for the same questions was about 80%. The grand mean for rating of questions in the cognitive, psychomotor and attitudes domain was 4.1, 3.9 and 4.2 out of 5 respectively. The graduates were found to be better off (45%), comparable to (50%) and less than (5%) when compared to other graduates who worked with the same consultants in the past. Results of three general questions inquiring about graduates' practice safety, overall standing and consultants' preference for Gezira graduates had a 95% positive rating. It is concluded that these results provide significant positive answers for some basic questions concerning credibility which are often raised about Gezira and similar schools with innovative medical curricula.     

Approaches to learning in a problem-based medical programme: a developmental study

In this study we investigate how the introductory phase of the problem-based medical programme in Maastricht affects the study methods of students. On the first day of the academic year, 142 men and women medical students completed the Short Inventory of Study Approaches and again at the end of the introductory period. The study indicates that these study methods are fostered by training in problem-based learning given during the introductory period.     

The limits of tutors' comfort zones with four integrated knowledge themes in a problem-based undergraduate medical curriculum (Interview study)

PURPOSE: To explore how a cohort of problem-based learning (PBL) tutors (with diverse medical and other content expertise) conceptualised their students' integrated learning agenda, particularly for content less familiar to the vast majority, such as public health elements. SETTING: Problem-based undergraduate medical curriculum, The University of Liverpool, in its first-ever year. PARTICIPANTS: Foundation PBL tutors for Year 1 students, Semester 1. METHOD: A cross-sectional, semistructured telephone interview study was undertaken during spring 1997, with open-ended questions about Semester 1 experience of the four main themes. Qualitative data were analysed inductively and iteratively for emerging patterns and instances. RESULTS: All 34 tutors responded. Of these, 26/34 (76%) were male and 23 (68%) were medically qualified. Towards the end of each approximately (median) 20-minute interview, reflecting on the curriculum themes, tutors mostly identified with the basic/clinical science theme (Structure and Function). Almost half articulated a clear division (implicitly or explicitly 'fact' versus 'non-fact') between it and the 'other three' themes of behavioural science, population science, and ethicolegal aspects of professional practice, respectively. Only 14/34 (41%) of tutors (including both public health doctors) outlined the public health-based theme adequately without disclosing confusion, antagonism/indifference or difficulties/uncertainty. CONCLUSION: This study provides baseline qualitative insights about new PBL tutors' insecurities when facilitating student discussion across integrated content. Given the difficulties of recruiting suitable educators into this role and potential resource limitations, staff retention and development strategies still must confront the reality of PBL tutors' bemusement when they are placed outwith their usual comfort zones.     

Longterm effects of problem-based learning: a comparison of competencies acquired by graduates of a problem-based and a conventional medical school

BACKGROUND: Problem-based learning (PBL) as an approach to the instruction of medical students has attracted much attention in recent years. However, its effect on the performance of its graduates is the subject of considerable debate. This article presents data from a large-scale study among graduates of a problem-based medical school and those of a conventional medical school to contribute to this discussion. PURPOSE: To study the longterm effects of problem-based medical training on the professional competencies of graduates. METHODS: A questionnaire was sent to all graduates since 1980 of a problem-based and a conventional medical school. Participants were requested to rate themselves on 18 professional competencies derived from the literature. RESULTS: The graduates of the PBL school scored higher on 14 of 18 professional competencies. Graduates of the problem-based school rated themselves as having much better interpersonal skills, better competencies in problem solving, self-directed learning and information gathering, and somewhat better task-supporting skills, such as the ability to work and plan efficiently. There were no sizeable differences with regard to general academic competencies, such as conducting research or writing a paper. Graduates from the conventional school rated themselves as having slightly more medical knowledge. The findings were shown to be valid and robust against possible response bias. CONCLUSION: The findings suggest that PBL not only affects the typical PBL-related competencies in the interpersonal and cognitive domains, but also the more general work-related skills that are deemed important for success in professional practice.     

General competencies of problem-based learning (PBL) and non-PBL graduates

INTRODUCTION: Junior doctors have reported shortcomings in their general competencies, such as organisational skills and teamwork. We explored graduates' perceptions of how well their training had prepared them for medical practice and in general competencies in particular. We compared the opinions of graduates from problem-based learning (PBL) and non-PBL schools, because PBL is supposed to enhance general competencies. METHOD: We analysed the responses of 1159 graduates from 1 PBL and 4 non-PBL schools to a questionnaire survey administered 18 months after graduation. RESULTS: Compared with their non-PBL colleagues, the PBL graduates gave higher ratings for the connection between school and work, their medical training and preparation for practice. According to the graduates, the most frequently used competencies with sufficient coverage during medical training were expert knowledge, profession-specific skills and communication skills. The majority of the PBL graduates, but less than half of the non-PBL graduates, indicated that communication skills had been covered sufficiently. All the graduates called for more curriculum attention on working with computers, planning and organisation, and leadership skills. More PBL graduates than non-PBL graduates indicated that they had learned profession-specific methods, communication skills and teamwork in medical school. DISCUSSION: Overall, the graduates appeared to be satisfied with their knowledge and skills. The results suggest that the PBL school provided better preparation with respect to several of the competencies. However, both PBL and non-PBL graduates identified deficits in their general competencies, such as working with computers and planning and organising work. These competencies should feature more prominently in undergraduate medical education.     

Effects of conventional and problem-based learning on clinical and general competencies and career development

Objective  To test hypotheses regarding the longitudinal effects of problem-based learning (PBL) and conventional learning relating to students' appreciation of the curriculum, self-assessment of general competencies, summative assessment of clinical competence and indicators of career development.
Methods  The study group included 2 complete cohorts of graduates who were admitted to the medical curriculum in 1992 (conventional curriculum, n  =   175) and 1993 (PBL curriculum, n  =   169) at the Faculty of Medicine, University of Groningen, the Netherlands. Data were obtained from student records, graduates' self-ratings and a literature search. Gender and secondary school grade point average (GPA) scores were included as moderator variables. Data were analysed by a stepwise multiple and logistic regression analysis.
Results  Graduates of the PBL curriculum scored higher on self-rated competencies. Contrary to expectations, graduates of the PBL curriculum did not show more appreciation of their curriculum than graduates of the conventional curriculum and no differences were found on clinical competence. Graduates of the conventional curriculum needed less time to find a postgraduate training place. No differences were found for scientific activities such as reading scientific articles and publishing in peer- reviewed journals. Women performed better on clinical competence than did men. Grade point average did not affect any of the variables.
Conclusions  The results suggest that PBL affects self-rated competencies. These outcomes confirm earlier findings. However, clinical competence measures did not support this finding.     

Building confidence for work as house officers: student experience in the final year of a new problem-based curriculum

INTRODUCTION: Newly qualified doctors require an appropriate level of confidence for their new roles. Development of this confidence was a key objective in the final year of a new integrated course with an emphasis on student self-direction. CONTEXT: There are 5 placements in the final year course. Students use a Learning Planner to help them choose suitable placements and objectives to serve their learning needs. Educational supervision focuses on helping students determine their objectives and assessing them against these. METHODS: Course evaluation was by means of a questionnaire during final assessments. Cohorts of 310 and 316 students in successive years completed the evaluation. The interrelationship between variables was explored using logistic regression. RESULTS: 220/310 students in the 2000 cohort and 214/316 in the 2001 cohort agreed they felt confident with their prospective role as a pre-registration house officer (PRHO). Confidence was significantly associated with confidence in their clinical skills, belief in their ability to cope with uncertainty and feeling able to work as a team member. The experience of the 2 hospital placements and (in 2000 only) the elective was associated with increased confidence. In all placements helpful educational supervision and the achievement of the self-directed learning plan was associated with increased confidence as a potential doctor. CONCLUSION: Students perceive a relationship between learning experiences in the final year of a self-directed course and development of confidence for their future role. Whilst further elucidation of the nature of this relationship is required, this provides encouragement to curriculum planners to promote self-direction.     

The first batch of graduates of a new medical curriculum in Asia: how their teachers see them

OBJECTIVE: To examine teachers' views of the first batch of graduates of a revised medical curriculum in Asia. METHODS: A cross-sectional study using a structured questionnaire was carried out to obtain the views of all the clinical teachers involved in teaching final year students of the old curriculum in 2000-01 and the new curriculum in 2001-02 at the University of Hong Kong, which commenced curricular reform in 1997. RESULTS: Nearly 62% of respondents felt that better graduates were being produced with the new curriculum. The majority of them rated the new curriculum students better in nearly all the major goals of the new curriculum, such as self-directed learning initiative, problem solving skills, interpersonal skills and clinical performance in patient care. However, the core knowledge of the new curriculum students was of concern to some teachers. CONCLUSION: This study focused on the first complete cycle of a revised medical curriculum in Asia. Teachers' views of the new curriculum students were highly positive and they felt that better graduates were being produced.     

Development of clinical reasoning from the basic sciences to the clerkships: a longitudinal assessment of medical students' needs and self-perception after a transitional learning unit

BACKGROUND: To facilitate students' transition from basic, science-oriented, problem-based learning (PBL) to clinical reasoning-oriented PBL, the University of Geneva School of Medicine introduced a 12-week unit of Introduction to Clinical Reasoning (ICR) at the beginning of its fourth or clerkship year. PURPOSE: The aims of the present study were to determine, after 12 weeks in the ICR unit, to what extent students had: (1) identified the learning content set by the faculty while adapting to the hypothetico-deductive reasoning approach; (2) familiarised themselves with the clinical reasoning-oriented learning process, and (3) transferred and further developed this process during the clinical years. METHOD: Students' derived objectives from the problems were compared to the objectives preset by the faculty to determine acquisition of intended learning content. To assess their adaptation to the clinical reasoning-oriented PBL approach, students (n = 124) were asked to list and freely comment on aspects of the unit they felt most at ease with or had difficulty with, and to complete a questionnaire on the clinical reasoning process (CRP). The same questionnaire was administered 6 and 12 months later to assess the evolution of the students' self-perception during clerkships. RESULTS: On average, student objectives matched 62% of faculty objectives. Half of the missed (38%) objectives were in basic sciences. Students generated 16% additional objectives, also predominantly in the basic sciences category (41%). Free comments indicated that the difficulties perceived by students were very similar to those previously reported in studies on reasoning and errors, such as difficulty in gathering, interpreting and weighting relevant data, synthesising information, and organising it hierarchically. These results were confirmed with the CRP questionnaire administered at the end of the unit. For most of the competencies assessed on the CRP questionnaire, a gradual improvement was seen to have occurred by 6 and 12 months after the unit. CONCLUSIONS: To ease students' transition from the preclinical to clinical years, a learning unit should give them the opportunity to train their clinical reasoning processes on standardised and prototypical problems, before encountering real patients with more ill-structured problems during clerkships. Such a transitional structure should particularly emphasise a developed repertoire of problem representations, recognition of key findings and a hierarchical classification of working hypotheses. It should foster the creation of links between the acquired basic clinical knowledge and the diagnostic, management and therapy steps of problem solving.     

Self-directed learning in anatomy: incorporation of case-based studies into a conventional medical curriculum

This paper reports an attempt to develop self-directed learning skills in second-year medical students by introducing case-based projects into the gross anatomy course at a long-established medical school. The programme and students' responses to a questionnaire completed at the end of the year are presented. Information on the various resources used by students to find information is given. The performance of students in the case-based components of the course has been evaluated and also in the more traditional end-of-year written examination. The data confirm that students have recognized that the projects were about obtaining a deeper understanding of the anatomy, and the programme appears to have promoted the use and study of library texts.     

Inquiry: the pedagogical importance of a skill central to clinical practice

The skill of inquiry is central to the task of the doctor confronted with a patient problem. Despite its importance this skill is not given appropriate emphasis in the education of medical students or in research concerning the clinical reasoning skills of doctors.     

Foundations of problem-based learning: some explanatory notes

Summary. The present article elaborates on cognitive effects of problem-based learning put forward by Schmidt, De Volder, De Grave, Moust & Patel (1989) and Norman & Schmidt (1992) . Its purpose is to discuss, in some detail, the theoretical premises of this approach to learning and instruction. It is argued that problem-based learning, above all, promotes the activation of prior knowledge and its elaboration. Evidence is reviewed demonstrating that these processes actually occur in small-group tutorials and that the processing of new information is indeed facilitated by discussion of a relevant problem. These effects must be attributed to a reorganization taking place in the knowledge structures of students as a result of problem-oriented study. In addition, a cognitive process called epistemic curiosity (or intrinsic interest) is enabled. Some directions for further research are outlined. The contribution starts, however, with a discussion of the philosophical and pedagogical roots of problem-based learning.     

Impact of a new curriculum on medical students' interpersonal and interviewing skills

OBJECTIVES: In 1998 we reported on the rise and fall of medical student communication skills during the 4 years of medical school. Since then, the University of Connecticut School of Medicine has completed a major curriculum renewal project with an emphasis on early clinical work, lifelong learning and more ambulatory training. The goals of this study were to compare students' interviewing and interpersonal skills in standardised patient (SP) assessments in the old and new curricula and to assess the success of the new curriculum in preventing a decline in student skills in this domain. METHODS: The clinical skills of 202 students were measured longitudinally during encounters with SPs in each of their 4 years of medical school. Students in this study and the earlier study were evaluated using the Arizona Clinical Interviewing Rating (ACIR) Scale. RESULTS: Compared with students from the previous curriculum, students on the new curriculum in this study showed an improvement in ACIR scores. Year 1 mean ACIR scores (1 = poor to 5 = excellent) were, respectively, 3.6 for the old curriculum cohort and 4.0 for the new curriculum group. In Year 4 the mean score for the old curriculum cohort was 3.7 and that for the new curriculum group was 3.8. Students on the new curriculum still showed a decline in ACIR scores from Years 1 to 4, but it was not as severe a decline as it had been previously. CONCLUSIONS: Pre-clinical medical students perform better on measures of interpersonal communication than their clinical counterparts. The students who participated in the new curriculum demonstrated an earlier acquisition of and a less steep decline in interviewing and interpersonal skills during the course of medical school.