Assessment of problem-solving ability

It is recognized that no problem solving can take place without a background knowledge. As the highest marks in PMPs are now being consistently achieved by those candidates who arrive at the correct diagnosis without accumulating excessive information and at the same time avoiding irrelevant or incorrect data the PMP may be measuring efficiency in problem solving ability. This would appear to be substantiated by the fact that reasonable correlations are obtained between each candidate's remarks in problems of widely differing system disease patterns.
The mark allocation is such that the problem solving ability relates to the field of general practice and no detailed specialist knowledge of any particular disease pattern is being measured. In measuring problem solving ability it is advisable to allocate a maximum positive mark for each section, this mark being the total of that awarded for information considered necessary to solve the particular problem. Any negative marks scored must be taken from the maximum positive mark allowed and not from the total positive score obtained, which may be considerably more than the former. In this way the efficient performer is appropriately rewarded whilst the more devious data gatherer is appropriately penalized.     

Problems of research into medical problem solving: some remarks on theory and method

Investigators of medical problem solving, and of problem solving in general, appear to use the term problem solving to denote different concepts.
Medical problem solving sometimes refers to the doctor solving the patient's problem, whereas others use it to refer to the doctor solving his own diagnostic problem. Second, 'problem' is used by some in a subjective sense (something is only called a problem if the subject has difficulties with it), and by others in an objective sense (problem being nearly synonymous with task). Finally, there is the definitional question of whether knowledge and problem solving are regarded as independent or as intimately related.
What one means by problem solving, with one's research objectives (research may be aimed at medical education, medical practice or cognitive theory), constitute the major determinants of the choice of research design.
It is advocated that investigators, before selecting a research design, should clarify their own definitions and research objectives, and before adopting other investigators' definitions and research designs, should consider carefully their research objectives.     

Use of scheme-based problem solving: an evaluation of the implementation and utilization of schemes in a clinical presentation curriculum

CONTEXT: The University of Calgary has implemented a new curriculum which is organized according to 120 ways in which patients may present to a physician. Students are taught scheme-based problem solving rather than the more typical hypothetico-deductive or search and scan approach to problem resolution. OBJECTIVE: This study sought to determine the extent to which faculty and students were implementing and utilizing scheme-based problem solving. METHOD: All classes taught within the new clinical presentation curriculum were surveyed at the year end. Participants included four classes of first-year students and three classes of second-year students. Using a 5-point scale, students responded to survey items regarding scheme implementation and utilization. RESULTS: Data were analysed using MANOVA (multivariate analysis of variance) and revealed significant differences among classes in both first- and second-year students. Increments in scheme implementation and utilization by instructors and students were observed, although instructors' utilization of schemes lagged behind that of students. A levelling effect to the benefits of schemes for problem solving was also evident. First-year students reported schemes to be very useful for learning and organizing new information. CONCLUSION: Although it has taken time to implement curriculum change, the student response to schemes has been favourable. Faculty development and further generation of pictorial/spatial representations for all schemes, to ensure that all clinical presentations provide pathways that students can use for both learning and problem solving are recommended. Whether students who utilize schemes are more successful problem solvers is not yet known but will be the subject of study as soon as scheme delivery is predominant.     

The art and science of clinical teaching

A review of the literature on clinical teaching indicates that medical schools emphasize didactic teaching of facts. Interpersonal skills, problem solving skills and cultivation of attitudes are neglected. However, the literature also described the skills of 'good' clinical teachers, and enumerates them. Programmes attempting to develop students' interpersonal skills and problem solving skills are reported. Finally, areas for further practice and study in clinical teaching are recommended.     

Changing attitudes of medical teachers towards medical education

If medical education is to improve, this is largely dependent upon the attitudes of clinical teachers towards their own teaching and their students' learning. The aims and objectives of our workshop method of medical teacher training therefore includes some in the affective domain. Based on the theory of cognitive dissonance the experience-linked practical problem solving approach of the workshop encourages behaviour change, presuming this to cause 'attitude change'. Testing by means of a specially constructed and validated Likert-type attitude scale shows that attitudes do improve significantly and consistently over the 5 days of our workshops. A follow-up study shows these effects to be maintained and taken back to the participants' own medical schools.     

Learning decision-making in clinical medicine: a card game dealing with acute emergencies for undergraduate use

A card game concerning the assessment and management of patients involved in medical emergencies was constructed for use by medical students who were beginning Year 3 of a 5-year curriculum. Particular emphasis was placed on the identification by the student of priorities for action and the justification for decisions made on the basis of underlying mechanisms. Our aim was to help the student sharpen his problem-solving skills in situations where decisions may have to be made and interventions undertaken before information gathering is complete. The game is simple in construct and may be played by individuals or groups. It has a major advantage over other forms of problem presentation in that the student can select and document his own decision-making pathway with minimal cueing and he may compare his pathway with that of the experts at the end of the game. Students and tutors found this approach enjoyable and realistic and considered the game a useful tool in our problem solving curriculum.     

Clinical problem solving and clinical knowledge

Videotape and simulated patients were used for evaluating third year medical students' performances in solving familiar and unfamiliar clinical problems. To assist the doctor to rate the students' performances by the review of videotapes, criteria were set up for students' behaviours that could be construed as evidence that the students recognized cues obtained by interviewing the patient and the information content of such cues. Criteria for data use by the students were also set. A mean of 90% of the pertinent medical history data was collected by the students. The amount of pertinent historical data collected was found to correlate very closely with student-patient interaction analysis score for the familiar and unfamiliar problems (r = 0.89 and r = 0.92, respectively). Significant differences in mean scores for data generation (P less than 0.001) and for data use (P less than 0.01) were found between those students who solved and those who failed to solve the familiar problem. With the unfamiliar problem a significant difference was found between the mean scores for data generation (P less than 0.01), but not for data use. The implications of these findings in medical education are discussed.     

Simulated patients as a learning resource in the study of reproductive medicine

Simulated patients have been used to foster active learning and problem solving skills in an undergraduate programme of reproductive medicine. These simulators were trained to present a series of clinical problems and were used in a variety of approaches. They served to highlight communication skills and to force attention on the recognition, assessment and management of patient problems.     

The reliability of a hypothesis generation and testing task

The purpose of this paper is to present results of initial experience with a clinical reasoning task which assesses two clearly defined aspects of clinical problem solving. Fourteen senior and 40 junior medical students at the University of Michigan Medical School participated in this study. They were given three clinical reasoning problems--the hypothesis generation and testing tasks (HG & T). As suggested by the name, two specifically defined components of clinical problem-solving, developing the initial hypotheses or differential and then testing hypotheses, were evaluated by these tasks. The findings of this study indicate that hypothesis generation and testing can be reliably evaluated with between seven and ten tasks. The results of this study suggest that reliable assessments of specific components of clinical problem-solving can be developed.     

Problem-based learning: rationale and description

Problem-based learning is an instructional method that is said to provide students with knowledge suitable for problem solving. In order to test this assertion the process of problem-based learning is described and measured against three principles of learning: activation of prior knowledge, elaboration and encoding specificity. Some empirical evidence regarding the approach is presented.     

A case-based anatomy course

Summary. A course is described in which the students give consideration to clinical cases as they review their current knowledge, and follow a set protocol which guided them in preparing their learning. The students are required to examine a case history, put down as key words either their first answer or the knowledge they consider they would need to answer specific questions. They then select stations at which additional information, wet specimens, models and radiological images assist them in upgrading their knowledge. They complete the exercise by writing a final answer to the questions on the case. The tutor is able through examination of the key words to determine the knowledge of the students as they enter, and confirm appropriate learning by inspection of the final answer. The students are able to identify their own deficiencies, develop strategies for thinking and learning, resulting in the acquisition of expertise in problem solving, and extend their communication skills by working with colleagues. While the course was designed for second-year anatomy teaching it could be applied to other disciplines.     

Reasoning strategies and the use of biomedical knowledge by medical students

This paper reports one of a series of studies conducted to investigate the role of biomedical knowledge in clinical reasoning. It was motivated by findings from our earlier studies that demonstrate that when specific basic science information is provided to medical students prior to solving a clinical case, they are unable to use this information in explaining the patient problem. An experiment was designed to investigate the use of biomedical information in the explanation of a clinical problem without any basic science information (spontaneous explanation) and where basic science information was provided after the clinical case (biomedically primed explanation). The results are discussed in the context of a two-stage model of diagnostic reasoning. The first stage is referred to as data-driven reasoning, and is characterized by the triggering of inferences from observations in the data to hypotheses. The second stage is designated as predictive reasoning, and is characterized by the generation of inferences driven by hypotheses. The results show that, with the exception of final-year medical students, the use of biomedical information interfered with the data-driven reasoning process. However, it did facilitate the process of predictive reasoning by the students. It is proposed that a sound disease classification scheme is necessary before biomedical knowledge can facilitate both data-driven and predictive reasoning during clinical problem-solving.     

The selection and training of examiners for clinical examinations

The inconsistency of the marking in clinical examinations is a well documented problem. This project identified some of the factors responsible for this inconsistency.
A standardized rating situation was devised. Five students were videotaped as they performed part of a physical examination on simulated patients. Eighteen experienced medical and surgical examiners rated their performances using an objective checklist type of rating form. No differences were evident between physicians and surgeons. The group of examiners was divided into three subgroups, one receiving no training, one limited training and one more extensive training. Examiners re-rated the same students 2 months after the first rating.
Inter-rater reliability was satisfactory for the first ratings and training produced no significant improvement. A substantial improvement was achieved by identifying the most inconsistent raters and removing them from the analysis. Training was shown to be unnecessary for consistent examiners and ineffective for examiners who were less consistent. On the basis of these results, only consistent examiners were selected to take part in the interactive component of the objective structured final year examinations. The ratings in these examinations achieved high levels of inter-rater reliability.
It was concluded that the combination of an objective check-list rating form, a controlled test situation and the selection of inherently consistent examiners could solve the problem of inconsistent marking in clinical examinations.     

Knowledge and clinical problem-solving

A consistent finding in the literature on measures of clinical problem-solving scores is that there are very low correlations across different problems. This phenomenon is commonly labelled 'content-specificity', implying that the scores differ because the content knowledge necessary to solve the problems differs. The present study tests this hypothesis by presenting groups of residents and clinical clerks with a series of simulated patient problems in which content was systematically varied. Each subject also completed a multiple choice test with questions linked to each diagnosis presented in the clinical problems. Three of the four problem-solving scores showed low correlations, even to two presentations of the same problem, and no relationship to content differences. None of the scores were related to performance on the multiple choice test. The results suggest that variability in problem-solving scores is related to factors other than content knowledge, and several possibilities are discussed.     

Biomedical knowledge in explanations of clinical problems by medical students

This paper was motivated by a controversy concerning the role of basic sciences in medical education. A problem underlying this issue is that it is unknown how basic science is used in clinical reasoning. The experiment was designed to address this issue. Three texts were constructed dealing with basic science knowledge relevant to a clinical problem. Subjects were asked to read and recall the texts. Next, the subjects were required to read and recall the clinical text describing a patient problem. Finally, they were asked to provide a diagnosis and an explanation of the underlying pathophysiology. Subjects were first-, second- and fourth-year medical students. Detailed analysis of subjects' protocols are presented. In general, the results show that when basic science information is given before the clinical problem, the basic science knowledge is used either incorrectly or inconsistently in explaining the clinical problem by all subjects. The authors interpret these results to indicate that the basic sciences and the more practical clinical knowledge form two separate domains with their own individual structures and the clinical information cannot be embedded into the basic science knowledge structure.     

The attainment of specified objectives by medical students in different learning environments

Medical students were provided with behavioural objectives for the problem solving phase of their curriculum. They were assigned for 11 weeks to various hospital settings that differed in the presence or absence of structured classes, the expertise of the teaching staff, the amount of instruction time and the types of learning experiences. At the end of that time, the students took the same examination to measure attainment of the objectives. The only variable that was associated with significantly different examination scores was presence or absence of structured classes.     

The dialysis exercise: a learning simulation for medical students

The dialysis exercise is a small group decisionmaking clinical simulation which has been completely successful as an educational technique. The success of the exercise stems from its ability to engage students cognitively and affectively while teaching problem solving skills. This complete learning package is presented, together with comments on our experience, for use in other settings.