Growth of medical knowledge

BACKGROUND: Knowledge is an essential component of medical competence and a major objective of medical education. Thus, the degree of acquisition of knowledge by students is one of the measures of the effectiveness of a medical curriculum. We studied the growth in student knowledge over the course of Maastricht Medical School's 6-year problem-based curriculum. METHODS: We analysed 60 491 progress test (PT) scores of 3226 undergraduate students at Maastricht Medical School. During the 6-year curriculum a student sits 24 PTs (i.e. four PTs in each year), intended to assess knowledge at graduation level. On each test occasion all students are given the same PT, which means that in year 1 a student is expected to score considerably lower than in year 6. The PT is therefore a longitudinal, objective assessment instrument. Mean scores for overall knowledge and for clinical, basic, and behavioural/social sciences knowledge were calculated and used to estimate growth curves. FINDINGS: Overall medical knowledge and clinical sciences knowledge demonstrated a steady upward growth curve. However, the curves for behavioural/social sciences and basic sciences started to level off in years 4 and 5, respectively. The increase in knowledge was greatest for clinical sciences (43%), whereas it was 32% and 25% for basic and behavioural/social sciences, respectively. INTERPRETATION: Maastricht Medical School claims to offer a problem-based, student-centred, horizontally and vertically integrated curriculum in the first 4 years, followed by clerkships in years 5 and 6. Students learn by analysing patient problems and exploring pathophysiological explanations. Originally, it was intended that students' knowledge of behavioural/social sciences would continue to increase during their clerkships. However, the results for years 5 and 6 show diminishing growth in basic and behavioural/social sciences knowledge compared to overall and clinical sciences knowledge, which appears to suggest there are discrepancies between the actual and the planned curricula. Further research is needed to explain this.     

A discourse analysis study of 'good' and 'poor' communication in an OSCE: a proposed new framework for teaching students

BACKGROUND: There is still a great deal to be learnt about teaching and assessing undergraduate communication skills, particularly as formal teaching in this area expands. One approach is to use the summative assessments of these skills in formative ways. Discourse analysis of data collected from final year examinations sheds light on the grounds for assessing students as 'good' or 'poor' communicators. This approach can feed into the teaching/learning of communication skills in the undergraduate curriculum. SETTING: A final year UK medical school objective structured clinical examination (OSCE). METHODS: Four scenarios, designed to assess communication skills in challenging contexts, were included in the OSCE. Video recordings of all interactions at these stations were screened. A sample covering a range of good, average and poor performances were transcribed and analysed. Discourse analysis methods were used to identify 'key components of communicative style'. FINDINGS: Analysis revealed important differences in communicative styles between candidates who scored highly and those who did poorly. These related to: empathetic versus 'retractive' styles of communicating; the importance of thematically staging a consultation, and the impact of values and assumptions on the outcome of a consultation. CONCLUSION: Detailed discourse analysis sheds light on patterns of communicative style and provides an analytic language for students to raise awareness of their own communication. This challenges standard approaches to teaching communication and shows the value of using summative assessments in formative ways.     

Integrating human factors into the medical curriculum

Background  The study of human factors is a scientific discipline that deals with the interactions between human beings and the elements of a system. This is important because shortcomings in these areas, if unchecked, can result in adverse outcomes. Research into human factors in industries where safety is paramount has provided the basis of countermeasures against human error. Adverse outcomes in medicine resulting from human error exact a high cost in both patient suffering and financial outlay.
CRM training  One of the approaches used to minimise the effect of human error is to train people in a set of knowledge, skills and attitudes that underpin the domain-specific competencies for that profession. These are referred to as non-technical skills (NTS). In aviation, such an approach has been shown to be both translatable from the training environment to the workplace and effective in reducing adverse outcomes.
Discussion  Medicine has incorporated this style of training, usually centred around simulator-based courses, but as yet in a piecemeal, episodic fashion which relies on participants volunteering to attend courses. Unlike other industries there is no systematic approach to linking the content of this teaching with the more conventional range of topics. As a consequence it is difficult to assess the impact of human factors training in medicine. This is partly because very little work has been done to date in identifying the key non-technical skills required in medicine, and the overall experience of workplace based assessment is limited. Lessons from other high reliability organisations may help to address the main challenges of developing the content, integrating it into the curriculum, reinforcing the concepts in the workplace through staff development and establishing its role in summative assessment.     

Item response theory: applications of modern test theory in medical education

CONTEXT: Item response theory (IRT) measurement models are discussed in the context of their potential usefulness in various medical education settings such as assessment of achievement and evaluation of clinical performance. PURPOSE: The purpose of this article is to compare and contrast IRT measurement with the more familiar classical measurement theory (CMT) and to explore the benefits of IRT applications in typical medical education settings. SUMMARY: CMT, the more common measurement model used in medical education, is straightforward and intuitive. Its limitation is that it is sample-dependent, in that all statistics are confounded with the particular sample of examinees who completed the assessment. Examinee scores from IRT are independent of the particular sample of test questions or assessment stimuli. Also, item characteristics, such as item difficulty, are independent of the particular sample of examinees. The IRT characteristic of invariance permits easy equating of examination scores, which places scores on a constant measurement scale and permits the legitimate comparison of student ability change over time. Three common IRT models and their statistical assumptions are discussed. IRT applications in computer-adaptive testing and as a method useful for adjusting rater error in clinical performance assessments are overviewed. CONCLUSIONS: IRT measurement is a powerful tool used to solve a major problem of CMT, that is, the confounding of examinee ability with item characteristics. IRT measurement addresses important issues in medical education, such as eliminating rater error from performance assessments.     

Low- to high-fidelity simulation – a continuum of medical education?

Context  Changes in medical training and culture have reduced the acceptability of the traditional apprenticeship style training in medicine and influenced the growth of clinical skills training. Simulation is an educational technique that allows interactive, and at times immersive, activity by recreating all or part of a clinical experience without exposing patients to the associated risks. The number and range of commercially available technologies used in simulation for education of health care professionals is growing exponentially. These range from simple part-task training models to highly sophisticated computer driven models.
Aim  This paper will review the range of currently available simulators and the educational processes that underpin simulation training. The use of different levels of simulation in a continuum of training will be discussed. Although simulation is relatively new to medicine, simulators have been used extensively for training and assessment in many other domains, most notably the aviation industry. Some parallels and differences will be highlighted.     

Traditional finals and OSCEs in predicting consultant and self-reported clinical skills of PRHOs: a pilot study

INTRODUCTION: As we move from standard 'long case' final examinations to new objective structured formats, we need to ensure the new is at least as good as the old. Furthermore, knowledge of which examination format best predicts medical student progression and clinical skills development would be of value. METHODS: A group of medical students sat both the standard long case examination and the new objective structured clinical examination (OSCE) to introduce this latter examination to our Medical School for final MB. At the end of their pre-registration year, the group and their supervising consultants submitted performance evaluation questionnaires. RESULTS: Thirty medical students sat both examinations and 20 returned evaluation questionnaires. Of the 72 consultants approached, 60 (83%) returned completed questionnaires. No correlation existed between self- and consultant reported performance. The traditional finals examination was inversely associated with consultant assessment. Better performing students were not rated as better doctors. The OSCE (and its components) was more consistent and showed positive associations with consultant ratings across the board. DISCUSSION: Major discrepancies exist between the 2 examination formats, in data interpretation and practical skills, which are explicitly tested in OSCEs but less so in traditional finals. Standardised marking schemes may reduce examiner variability and discretion and weaken correlations across the 2 examinations. This pilot provides empirical evidence that OSCEs assess different clinical domains than do traditional finals. Additionally, OSCEs improve prediction of clinical performance as assessed by independent consultants. CONCLUSION: Traditional finals and OSCEs correlate poorly with one another. Objective structured clinical examinations appear to correlate well with consultant assessment at the end of the pre-registration house officer year.     

The rating of pre-clerkship examination questions by postgraduate medical students: an assessment of quality and relevancy to medical practice

OBJECTIVE: To determine whether postgraduate students are able to assess the quality of undergraduate medical examinations and to establish whether faculty can use their results to troubleshoot the curriculum in terms of its content and evaluation. SUBJECTS: First and second year family medicine postgraduate students. MATERIALS: A randomly generated sample of undergraduate medical examination questions. METHODS: Postgraduate students were given two undergraduate examinations which included questions with an item difficulty (ID) > 0.60. The students answered and then rated each question on a scale of 1-7. RESULTS: The percentage of postgraduate students answering each question correctly correlated significantly with the average perceived relevance (Examination 1: r=0.372; P < 0.05; Examination 2: r=0.458; P < 0.05). Questions plotted for average postgraduate/undergraduate performance ratio versus the average perceived relevance were significantly correlated (Examination 1: r=0.462; P < 0.01; Examination 2: r=0.458; P < 0.05). CONCLUSIONS: This study offers a method of validating question appropriateness prior to examination administration. The design has the potential to be used as a model for determining the relevancy of a medical curriculum.     

Factors in faculty evaluation of medical students' performance

CONTEXT: Faculty members often use global rating scales as a method of assessing various characteristics of medical students' clinical performance. The purpose of this study was to determine if some performance characteristics are more highly associated with the overall faculty grade than others. METHODS: The clinical performance of 211 surgery clerkship students was evaluated by 2 or 3 faculty preceptors. Faculty rated students on 10 specific performance characteristics, using a 5-point scale. Faculty then assigned a numerical grade summarising the faculty's view of the student's performance. Reliability of the ratings was estimated by the intraclass correlation, and 1-way (analysis of variance) anova was used to test for differences among the students' mean ratings. Logistic regression was employed to determine the accuracy of each performance measure in predicting students' grades (A or B). Stepwise logistic regression was used to determine if there was a combination of performance characteristics that best predicted students' grades. RESULTS: The inter-rater reliabilities were low (相似文献   

The problem with outcomes-based curricula in medical education: insights from educational theory

BACKGROUND: Educators across the world are charged with the responsibility of producing core learning outcomes for medical curricula. However, much educational theory exists which deliberates the value of learning outcomes in education. AIMS: This paper aims to discuss the problems surrounding outcomes-based curricula in medical education, using insights from educational theory. DISCUSSION: The paper begins with a discussion of the traditions, values and ideologies of medical curricula. It continues by analysing the issue of control within the curriculum and argues that curriculum designers and teachers control product-orientated curricula, leading to student disempowerment. The paper debates outcomes-based curricula from an ideological perspective and argues that learning outcomes cannot specify exactly what is to be achieved as a result of learning. CONCLUSIONS: The paper argues that medical schools should adopt a model for co-operative control of the curriculum, thus empowering learners. The paper also suggests that medical educators should determine the value of precise learning outcomes before blindly adopting an outcomes-based model.     

Teaching clinical skills to medical students: the use of simulated patients and videotaping in general practice

This paper describes the use of simulated patients in a problem-based teaching programme designed to develop the clinical skills of medical students during a 5-week course in general practice in the University of Leicester. The logistics of training and the advantages of using simulated patients are discussed. Simulated patients are practicable and acceptable in a United Kingdom medical school, and the reaction of students to their use is favourable.     

An acute care skills evaluation for graduating medical students: a pilot study using clinical simulation

PURPOSE: This investigation aimed to explore the measurement properties of scores from a patient simulator exercise. METHODS: Analytic and holistic scores were obtained for groups of medical students and residents. Item analysis techniques were used to explore the nature of specific examinee actions. Interrater reliability was calculated. Scores were contrasted for third year medical students, fourth year medical students and emergency department residents. RESULTS: Interrater reliabilities for analytic and holistic scores were 0.92 and 0.81, respectively. Based on item analysis, proper timing and sequencing of actions discriminated between low- and high-ability examinees. In general, examinees with more advanced training obtained higher scores on the simulation exercise. CONCLUSION: Reliable and valid measures of clinical performance can be obtained from a trauma simulation provided that care is taken in the development and scoring of the scenario.     

Responding to the implications of the genetics revolution for the education and training of doctors: a medical humanities approach

BACKGROUND: Rapid advances in the field of genetics continue to present medical educators with significant challenges. Whilst there is undoubtedly a pressing need to educate doctors about genetic disorders, research and therapies, there is a parallel need to provide a context for all of these. CONTENT: An interdisciplinary, arts and humanities based approach, responding to this need, is described. This teaching has been successfully delivered both as optional and core undergraduate teaching, and as part of continuing professional development. THE HUMAN PERSPECTIVE: STORIES NOT HISTORIES: Understanding of the patient's perspective can be significantly improved by drawing on both written and oral stories of illness. THE HISTORICAL PERSPECTIVE: LEARNING FROM THE PATIENT: Experiential learning provides insights into the social history of developments in genetics, thereby placing the current concern and debate about the new genetics in context. THE ROLE OF THE MEDIA: THE POWER TO PERSUADE: Critical reading skills can be developed and the power of the popular press to influence the reader acknowledged by analysing and employing the skills of the journalist when reporting developments in biotechnology. LEARNER ASSESSMENT AND EVALUATION: Assessment, both formative and summative, demonstrates sophisticated insights and perspectives into the lived experience of genetic illness. Learner evaluation of the teaching is high. CONCLUSION: Medical humanities offers a powerful way to convey an understanding of how genetic disorders impact on the lives of patients and families, and to set this against the background of a history rich in the uses, and abuses, of knowledge of heredity.     

Relevant behavioural and social science for medical undergraduates: a comparison of specialist and non-specialist educators

AIM: To compare what medical educators who are specialists in the behavioural and social sciences and their non-specialist counterparts consider to be core concepts that medical graduates should understand. BACKGROUND: Previously perceived as 'nice to know' rather than 'need to know', the General Medical Council (GMC) now places behavioural and social sciences on the same need-to-know basis as clinical and basic sciences. Attempts have been made to identify what components of these topics medical students need to know; however, it remains unknown if decisions over programme content differ depending on whether or not educationalists have specialist knowledge of the behavioural and social sciences. METHODS: In a survey of medical educationalists within all UK medical schools, respondents were asked to indicate from a comprehensive list of psychological, sociological and anthropological concepts what they considered a minimally competent graduate should understand. Comparisons were made between the concepts identified by specialist behavioural and social science (BSS) educators and those without such training. RESULTS: Despite different disciplinary backgrounds, non-specialist educators largely concurred with BSS specialist educators in the concepts they considered tomorrow's doctors should know about. However, among BSS specialists there remained disagreement on what BSS content was relevant for graduates. Differences reflect specialist knowledge and recognition of the role of theoretical underpinning of BSS and reveal gaps in non-specialists knowledge. CONCLUSIONS: Educationalists with formal training in the full range of behavioural and social sciences should be involved in the development of BSS curriculum content at both national and school levels.     

Teaching psychiatric interview skills to medical students

A teaching package is described for teaching interview skills to large blocks of medical students whilst on their psychiatric attachment. The aims of this package are to reduce students' concerns about interviewing psychiatric patients, to reinforce students' knowledge of basic interviewing skills and to introduce students to the particular skills required in taking a psychiatric history and mental state examination. The package emphasizes the following teaching methods: 'hands-on' experience of interviewing a patient in front of small groups of peers; peer feedback using check-lists which focus on three major aspects of interviewing; elicitation of facts, elicitation of feelings and control of the interview; facilitation of small-group discussions in the presence of senior psychiatrist. The active involvement of all students in interviewing psychiatric patients engages them in the learning process. Peer involvement increases motivation and was deemed by students as a supportive and constructive exercise. The presence of a senior psychiatrist ensures that discussion is focused on the process of interviewing rather than on patient pathology. Ideally this package would precede focused training throughout the subsequent psychiatric placement.     

Portfolio-based learning: continuing medical education for general practitioners — a mid-point evaluation

In October 1994 a project was initiated by the General Practice Continuing Medical Education Tutors in the Department of General Practice at Sheffield University. The project sought to evaluate the efficiency (effort expended) and effectiveness (distance travelled) of a model of continuing professional development for general practitioners through individual portfolio-based learning in co-mentoring groups. Learning demonstrated through the portfolio was accredited for the postgraduate education allowance of participants. This paper addresses the process of portfolio development at the mid-point of a year-long trial to ascertain the strengths, weaknesses and possible future development of such a process within the context of continuing medical education.     

Effectiveness of medical school admissions criteria in predicting residency ranking four years later

BACKGROUND: Medical schools across Canada expend great effort in selecting students from a large pool of qualified applicants. Non-cognitive assessments are conducted by most schools in an effort to ensure that medical students have the personal characteristics of importance in the practice of Medicine. We reviewed the ability of University of Toronto academic and non-academic admission assessments to predict ranking by Internal Medicine and Family Medicine residency programmes. METHODS: The study sample consisted of students who had entered the University of Toronto between 1994 and 1998 inclusive, and had then applied through the Canadian resident matching programme to positions in Family or Internal Medicine at the University of Toronto in their graduating year. The value of admissions variables in predicting medical school performance and residency ranking was assessed. RESULTS: Ranking in Internal Medicine correlated significantly with undergraduate grade point average (GPA) and the admissions non-cognitive assessment. It also correlated with 2-year objective structured clinical examination (OSCE) score, clerkship grade in Internal Medicine, and final grade in medical school. Ranking in Family Medicine correlated with the admissions interview score. It also correlated with 2nd-year OSCE score, clerkship grade in Family Medicine, clerkship ward evaluation in Internal Medicine and final grade in medical school. DISCUSSION: The results of this study suggest that cognitive as well as non-cognitive factors evaluated during medical school admission are important in predicting future success in Medicine. The non-cognitive assessment provides additional value to standard academic criteria in predicting ranking by 2 residency programmes, and justifies its use as part of the admissions process.     

Views of junior doctors on the specialist registrar (SpR) training scheme: qualitative study of UK medical graduates

OBJECTIVE: To report junior doctors' views on specialist registrar (SpR) training. DESIGN: In 1999, as part of ongoing studies of doctors' careers, we surveyed all doctors who had qualified from UK medical schools in 1993. Structured questions about recipients' careers were accompanied by a form inviting free text comments. Comments about the SpR scheme were extracted for analysis. RESULTS: Doctors commented that there were insufficient national training numbers (NTNs) for those who wanted them, and that more than the minimum entry requirements seemed necessary for shortlisting. Strengthening curricula vitae through research and published work could prolong the duration of training and did not guarantee success. Specialist registrar training was considered by some respondents to be narrow and inflexible, with service work taking priority over training needs. As a result, some respondents feared they would not be competent to practise as consultants. There was a perceived shortage of consultant vacancies and 6 months was considered insufficient time for obtaining a suitable post. DISCUSSION: It is inevitable that doctors may not necessarily be able to pursue their initially chosen career paths. Trainers need to provide realistic advice about career opportunities. Provision of information about NTN availability and formal career counselling could help to prevent delays in career progression. The shortened and more structured programme of training has reduced its flexibility in some doctors' experience. Improvements in educational content will need greater input from consultants, which may require an increase in consultant posts. Time will tell whether concerns about competence to practise as consultants and consultant post availability will be justified.