Monkey see, monkey do: a critique of the competency model in graduate medical education

BACKGROUND: Graduate medical education in the UK is in danger of being subsumed in a minimalist discourse of competency. ARGUMENT: While accepting that competence in a doctor is a sine qua non, the author criticises the construction of a graduate and specialist medical education based solely upon a competency model. Many competency models follow the concepts of either academic competence or operational competence, both of which have lately been subject to criticism. CONCLUSION: The author discusses the need for replacing such criterion-referenced models in favour of a model that engages the higher order competence, performance and understanding which represent professional practice at its best.     

Low- to high-fidelity simulation – a continuum of medical education?

Context  Changes in medical training and culture have reduced the acceptability of the traditional apprenticeship style training in medicine and influenced the growth of clinical skills training. Simulation is an educational technique that allows interactive, and at times immersive, activity by recreating all or part of a clinical experience without exposing patients to the associated risks. The number and range of commercially available technologies used in simulation for education of health care professionals is growing exponentially. These range from simple part-task training models to highly sophisticated computer driven models.
Aim  This paper will review the range of currently available simulators and the educational processes that underpin simulation training. The use of different levels of simulation in a continuum of training will be discussed. Although simulation is relatively new to medicine, simulators have been used extensively for training and assessment in many other domains, most notably the aviation industry. Some parallels and differences will be highlighted.     

Factors predictive of successful learning in postgraduate medical education

PURPOSE: To establish which personal and contextual factors are predictive of successful outcomes in postgraduate medical education. METHOD: We performed a follow-up study of 118 doctors on a postgraduate occupational health training programme on the management of mental health problems. The following personal and contextual variables were measured as potential predictors of outcome: gender; age; years of experience as a doctor; university of graduation; learning style (Kolb); present employer (occupational health service), and educational format (problem-based or lecture-based). The main outcome measures were: scores on knowledge tests consisting of true/false and open answer questions, and performance in practice based on self-report and performance indicators. To determine the effect of potential predictive factors univariate analyses of variance and repeated measurement analysis of variance was applied. RESULTS: The mean scores of knowledge (P < 0.001) and performance (P = 0.001) of the participants increased after the educational programme. After multivariate analysis female gender was positively related to accruements in both knowledge and performance (both P < 0.05), independently of the influence of other factors. Accommodator learning style showed a relation with knowledge increase (P = 0.05), but had no influence on performance (P = 0.79). The problem-based educational format yielded a better performance outcome (P = 0.05), but had no influence on knowledge tests (P = 0.31). CONCLUSION: Gender and learning style were found to be related to an increase in knowledge. Gender was also found to be related to improvement in performance after a postgraduate medical education programme. We found no interactions with course design (i.e. problem-based or non problem-based learning formats), but further research could reveal other cues, suggesting practical consequences of student characteristics for course design in postgraduate training.     

The acquisition of tacit knowledge in medical education: learning by doing

AIM: This study was designed to assess medical school teachers' tacit knowledge of basic pedagogic principles and to explore the specific character of the knowledge base. METHODS: We developed a 50-item, multiple-choice question test based on important pedagogic principles, and classified all questions as requiring either declarative or procedural knowledge. A total of 72 medical teachers representing 5 different groups of clinicians and educators agreed to sit the test. RESULTS: Teachers in all 5 groups performed well on the test of tacit pedagogic knowledge but those with advanced education degrees, or local recognition as experts, performed best. All test takers performed best on questions requiring procedural knowledge. CONCLUSION: Medical teachers possess tacit knowledge of basic pedagogic principles. Superior test performance on questions requiring procedural knowledge is consistent with their working in a clinical environment characterised by repeated procedural activities.     

Country report: medical education in France

CONTEXT: The last 10 years have represented a period of significant reform within both the health care and education systems in France. In terms of its workforce, France faces a shortage of doctors, particularly in primary care. METHODS: This paper examines the French medical curriculum, student selection, licensure and continuing medical education and discusses the challenges currently facing French medical faculties. RESULTS: The French medical curriculum is defined nationally, with methods adapted at medical school level. There has been some uptake of innovative methods such as problem-based learning, skills-based teaching and performance-based assessment. However, traditional didactic teaching of scientific medicine and the apprenticeship model remain dominant. France uses a unique method of selection, which is the subject of much debate. Following a general year, medical students are subject to a selection examination that permits only a small number to continue studies. Similarly, at the end of medical school, a written test is used to rank students for the purpose of matching to specialty training. France has no national colleges or licensing authorities and thus authorisation to practise rests on the diploma delivered by each faculty of medicine. From 2005, continuing medical education became compulsory for all doctors. It includes the evaluation of medical practice. CONCLUSIONS: French faculties of medicine face several challenges, including: rising numbers of students without a commensurate growth in the number of faculty members; an increasing emphasis on multidisciplinary health care; a drive towards mandatory continuing education and performance-based outcomes, and the development of national selection examinations that are knowledge-based.     

Growth of medical knowledge

BACKGROUND: Knowledge is an essential component of medical competence and a major objective of medical education. Thus, the degree of acquisition of knowledge by students is one of the measures of the effectiveness of a medical curriculum. We studied the growth in student knowledge over the course of Maastricht Medical School's 6-year problem-based curriculum. METHODS: We analysed 60 491 progress test (PT) scores of 3226 undergraduate students at Maastricht Medical School. During the 6-year curriculum a student sits 24 PTs (i.e. four PTs in each year), intended to assess knowledge at graduation level. On each test occasion all students are given the same PT, which means that in year 1 a student is expected to score considerably lower than in year 6. The PT is therefore a longitudinal, objective assessment instrument. Mean scores for overall knowledge and for clinical, basic, and behavioural/social sciences knowledge were calculated and used to estimate growth curves. FINDINGS: Overall medical knowledge and clinical sciences knowledge demonstrated a steady upward growth curve. However, the curves for behavioural/social sciences and basic sciences started to level off in years 4 and 5, respectively. The increase in knowledge was greatest for clinical sciences (43%), whereas it was 32% and 25% for basic and behavioural/social sciences, respectively. INTERPRETATION: Maastricht Medical School claims to offer a problem-based, student-centred, horizontally and vertically integrated curriculum in the first 4 years, followed by clerkships in years 5 and 6. Students learn by analysing patient problems and exploring pathophysiological explanations. Originally, it was intended that students' knowledge of behavioural/social sciences would continue to increase during their clerkships. However, the results for years 5 and 6 show diminishing growth in basic and behavioural/social sciences knowledge compared to overall and clinical sciences knowledge, which appears to suggest there are discrepancies between the actual and the planned curricula. Further research is needed to explain this.     

The problem with outcomes-based curricula in medical education: insights from educational theory

BACKGROUND: Educators across the world are charged with the responsibility of producing core learning outcomes for medical curricula. However, much educational theory exists which deliberates the value of learning outcomes in education. AIMS: This paper aims to discuss the problems surrounding outcomes-based curricula in medical education, using insights from educational theory. DISCUSSION: The paper begins with a discussion of the traditions, values and ideologies of medical curricula. It continues by analysing the issue of control within the curriculum and argues that curriculum designers and teachers control product-orientated curricula, leading to student disempowerment. The paper debates outcomes-based curricula from an ideological perspective and argues that learning outcomes cannot specify exactly what is to be achieved as a result of learning. CONCLUSIONS: The paper argues that medical schools should adopt a model for co-operative control of the curriculum, thus empowering learners. The paper also suggests that medical educators should determine the value of precise learning outcomes before blindly adopting an outcomes-based model.     

A web-based presentation of an undergraduate clinical skills curriculum

AIMS: The aim of this study was to use information and communications technology to present a curriculum of clinical skills in a user-friendly format. SETTING: A UK undergraduate medical school with a problem-based curriculum and a strong emphasis on proficiency in clinical skills. STUDY DESIGN: Case study describing the qualitative analysis of users' requirements and development of a web-based learning portfolio. EVALUATION: The study involved direct observation of users during a 'think-aloud' protocol, a validated software users' measurement inventory and a 17-item questionnaire designed to test whether 'SkillsBase' met its users' requirements. RESULTS: Students wanted a clear and flexible presentation of their skills curriculum that was easy to navigate, offered instructional material and standards for self- and peer assessment, offered useful Internet links, allowed them to compare their progress with school standards and peer norms, and could be used as a learning portfolio. During the think-aloud protocol, students made very few errors in data interpretation or navigation, and found SkillsBase easy to learn and aesthetically pleasing to use. They rated it higher on all measures of usability than standard commercial software. The questionnaire showed that it met most aspects of its design specification, although many students were doubtful that they would use its reflective function. It is available for inspection at http://www.skillsbase.man.ac.uk/. CONCLUSIONS: SkillsBase meets the design specification for a training and reflective aid to learning clinical skills and is very usable.     

Item response theory: applications of modern test theory in medical education

CONTEXT: Item response theory (IRT) measurement models are discussed in the context of their potential usefulness in various medical education settings such as assessment of achievement and evaluation of clinical performance. PURPOSE: The purpose of this article is to compare and contrast IRT measurement with the more familiar classical measurement theory (CMT) and to explore the benefits of IRT applications in typical medical education settings. SUMMARY: CMT, the more common measurement model used in medical education, is straightforward and intuitive. Its limitation is that it is sample-dependent, in that all statistics are confounded with the particular sample of examinees who completed the assessment. Examinee scores from IRT are independent of the particular sample of test questions or assessment stimuli. Also, item characteristics, such as item difficulty, are independent of the particular sample of examinees. The IRT characteristic of invariance permits easy equating of examination scores, which places scores on a constant measurement scale and permits the legitimate comparison of student ability change over time. Three common IRT models and their statistical assumptions are discussed. IRT applications in computer-adaptive testing and as a method useful for adjusting rater error in clinical performance assessments are overviewed. CONCLUSIONS: IRT measurement is a powerful tool used to solve a major problem of CMT, that is, the confounding of examinee ability with item characteristics. IRT measurement addresses important issues in medical education, such as eliminating rater error from performance assessments.     

Informal learning in postgraduate medical education: from cognitivism to 'culturism'

BACKGROUND: Work-based learning occupies a central role in the training and ongoing development of the medical workforce. With this arises the need to understand the processes involved, particularly those relating to informal learning. Approaches to informal learning in postgraduate medical education have tended to consider the mind as an independent processor of information. METHOD: In this paper, such cognitive approaches are critiqued and an alternative socio-cultural view on informal learning described. Recent and imminent changes in postgraduate medical education are identified, namely the reduction in patient experience, the fragmentation of teaching, and the development of competency frameworks and structured curricula. It is argued that although the latter may be useful in the construction of formal learning programmes, they will do little to enhance the progression of the individual from newcomer to old-timer or the cultural assimilation of the learner into a profession. DISCUSSION: Strategies for enhancing informal learning in the workplace are recommended in which increased attention is paid to the development of the medical apprentice within a community of social practice. These include the establishment of strong goals, the use of improvised learning practices, attention to levels of individual engagement and workplace affordances, immersion in professional discourse and behaviours, support in relation to the development of a professional identity and the provision of opportunities to transform social practice.     

Integrating human factors into the medical curriculum

Background  The study of human factors is a scientific discipline that deals with the interactions between human beings and the elements of a system. This is important because shortcomings in these areas, if unchecked, can result in adverse outcomes. Research into human factors in industries where safety is paramount has provided the basis of countermeasures against human error. Adverse outcomes in medicine resulting from human error exact a high cost in both patient suffering and financial outlay.
CRM training  One of the approaches used to minimise the effect of human error is to train people in a set of knowledge, skills and attitudes that underpin the domain-specific competencies for that profession. These are referred to as non-technical skills (NTS). In aviation, such an approach has been shown to be both translatable from the training environment to the workplace and effective in reducing adverse outcomes.
Discussion  Medicine has incorporated this style of training, usually centred around simulator-based courses, but as yet in a piecemeal, episodic fashion which relies on participants volunteering to attend courses. Unlike other industries there is no systematic approach to linking the content of this teaching with the more conventional range of topics. As a consequence it is difficult to assess the impact of human factors training in medicine. This is partly because very little work has been done to date in identifying the key non-technical skills required in medicine, and the overall experience of workplace based assessment is limited. Lessons from other high reliability organisations may help to address the main challenges of developing the content, integrating it into the curriculum, reinforcing the concepts in the workplace through staff development and establishing its role in summative assessment.     

Educational impact of in-training assessment (ITA) in postgraduate medical education: a qualitative study of an ITA programme in actual practice

Objectives To investigate the experiences and opinions of programme directors, clinical supervisors and trainees on an in‐training assessment (ITA) programme on a broad spectrum of competence for first year training in anaesthesiology. How does the programme work in practice and what are the benefits and barriers? What are the users' experiences and thoughts about its effect on training, teaching and learning? What are their attitudes towards this concept of assessment? Methods Semistructured interviews were conducted with programme directors, supervisors and trainees from 3 departments. Interviews were audiotaped and transcribed. The content of the interviews was analysed in a consensus process among the authors. Results The programme was of benefit in making goals and objectives clear, in structuring training, teaching and learning, and in monitoring progress and managing problem trainees. There was a generally positive attitude towards assessment. Trainees especially appreciated the coupling of theory with practice and, in general, the programme inspired an academic dialogue. Issues of uncertainty regarding standards of performance and conflict with service declined over time and experience with the programme, and departments tended to resolve practical problems through structured planning. Discussion Three interrelated factors appeared to influence the perceived value of assessment in postgraduate education: (1) the link between patient safety and individual practice when assessment is used as a licence to practise without supervision rather than as an end‐of‐training examination; (2) its benefits to educators and learners as an educational process rather than as merely a method of documenting competence, and (3) the attitude and rigour of assessment practice.     

Effect of practice on standardised learning outcomes in simulation-based medical education

OBJECTIVES: This report synthesises a subset of 31 journal articles on high-fidelity simulation-based medical education containing 32 research studies drawn from a larger qualitative review published previously. These studies were selected because they present adequate data to allow for quantitative synthesis. We hypothesised an association between hours of practice in simulation-based medical education and standardised learning outcomes measured as weighted effect sizes. METHODS: Journal articles were screened using 5 exclusion and inclusion criteria. Response data were extracted and 3 judges independently coded each study. Learning outcomes were standardised using a common metric, the average weighted effect size (AWES), due to the heterogeneity of response measures in individual studies. anova was used to evaluate AWES differences due to hours of practice on a high-fidelity medical simulator cast in 5 categories. The eta squared (eta2) statistic was used to assess the association between AWES and simulator practice hours. RESULTS: There is a strong association (eta2=0.46) between hours of practice on high-fidelity medical simulators and standardised learning outcomes. The association approximates a dose-response relationship. CONCLUSIONS: Hours of high-fidelity simulator practice have a positive, functional relationship with standardised learning outcomes in medical education. More rigorous research methods and more stringent journal editorial policies are needed to advance this field of medical education research.     

The theory of planned behaviour in medical education: a model for integrating professionalism training

Context  Teaching and evaluating professionalism remain important issues in medical education. However, two factors hinder attempts to integrate curricular elements addressing professionalism into medical school training: there is no common definition of medical professionalism used across medical education, and there is no commonly accepted theoretical model upon which to integrate professionalism into the curriculum.
Objectives  This paper proposes a definition of professionalism, examines this definition in the context of some of the previous definitions of professionalism and connects this definition to the attitudinal roots of professionalism. The problems described above bring uncertainty about the best content and methods with which to teach professionalism in medical education. Although various aspects of professionalism have been incorporated into medical school curricula, content, teaching and evaluation remain controversial. We suggest that intervening variables, which may augment or interfere with medical students' implementation of professionalism knowledge, skills and, therefore, attitudes, may go unaddressed.
Discussion  We offer a model based on the theory of planned behaviour (TPB), which describes the relationships of attitudes, social norms and perceived behavioural control with behaviour. It has been used to predict a wide range of behaviours, including doctor professional behaviours. Therefore, we propose an educational model that expands the TPB as an organisational framework that can integrate professionalism training into medical education. We conclude with a discussion about the implications of using this model to transform medical school curricula to develop positive professionalism attitudes, alter the professionalism social norms of the medical school and increase students' perceived control over their behaviours.     

Clinical teachers and problem-based learning: a phenomenological study

AIM: To explore how clinicians perceive their roles in problem-based medical education, and how closely those perceptions link to the curriculum they teach. METHOD: All 14 general physicians in a teaching hospital took part in 6 semistructured discussions, which were analysed phenomenologically. RESULTS: Third year clinical teaching was described in terms that bore little relation to problem-based learning (PBL). Teachers placed great importance on the social dimension of professional learning. They expressed strongly positive affects towards learners and their learning that they found hard to express as PBL tutors. Their narratives of education were remarkably divorced from modern day clinical practice. CONCLUSIONS: Problem-based method lacked some important conditions for professional teaching and learning. Traditional apprenticeship is unsustainable under present day conditions of practice. There is a need for new educational methods that help the learner to build a professional identity through social interaction with practitioners.     

The Bologna Process and health science education: times are changing

CONTEXT: The Bologna Declaration of 1999 has led to the initiation of profound changes within the European Higher Education Area. METHODS: This paper gives an overview of the Bologna Process, reviews the developments within European dental education that have arisen as a result of the Process, and describes the impact of the continuation of the Process on health science education in the UK. CONCLUSIONS: The Bologna Process has led to the establishment of the European Higher Education Area, which is committed to implementing a series of action lines by 2010. Currently, we are just over halfway to 2010, and the importance of UK higher education institutions engaging with the Bologna Process is stressed.     

Towards valid measures of self-directed clinical learning

AIM: To compare the validity of different measures of self-directed clinical learning. METHODS: We used a quasi-experimental study design. The measures were: (1) a 23-item quantitative instrument measuring satisfaction with the learning process and environment; (2) free text responses to 2 open questions about the quality of students' learning experiences; (3) a quantitative, self-report measure of real patient learning, and (4) objective structured clinical examination (OSCE) and progress test results. Thirty-three students attached to a single firm during 1 curriculum year in Phase 2 of a problem-based medical curriculum formed an experimental group. Thirty-one students attached to the same firm in the previous year served as historical controls and 33 students attached to other firms within the same module served as contemporary controls. After the historical control period, experimental group students were exposed to a complex curriculum intervention that set out to maximise appropriate real patient learning through increased use of the outpatient setting, briefing and supported, reflective debriefing. RESULTS: The quantitative satisfaction instrument was insensitive to the intervention. In contrast, the qualitative measure recorded a significantly increased number of positive statements about the appropriateness of real patient learning. Moreover, the quantitative self-report measure of real patient learning found high levels of appropriate learning activity. Regarding outpatient learning, the qualitative and quantitative real patient learning instruments were again concordant and changed in the expected direction, whereas the satisfaction measure did not. An incidental finding was that, despite all attempts to achieve horizontal integration through simultaneously providing community attachments and opening up the hospital for self-directed clinical learning, real patient learning was strongly bounded by the specialty interest of the hospital firm to which students were attached. Assessment results did not correlate with real patient learning. CONCLUSIONS: Both free text responses and students' quantitative self-reports of real patient learning were more valid than a satisfaction instrument. One explanation is that students had no benchmark against which to rate their satisfaction and curriculum change altered their tacit benchmarks. Perhaps the stronger emphasis on self-directed learning demanded more of students and dissatisfied those who were less self-directed. Results of objective, standardised assessments were not sensitive to the level of self-directed, real patient learning. Despite an integrated curriculum design that set out to override disciplinary boundaries, students' learning remained strongly influenced by the specialty of their hospital firm.     

The limits of tutors' comfort zones with four integrated knowledge themes in a problem-based undergraduate medical curriculum (Interview study)

PURPOSE: To explore how a cohort of problem-based learning (PBL) tutors (with diverse medical and other content expertise) conceptualised their students' integrated learning agenda, particularly for content less familiar to the vast majority, such as public health elements. SETTING: Problem-based undergraduate medical curriculum, The University of Liverpool, in its first-ever year. PARTICIPANTS: Foundation PBL tutors for Year 1 students, Semester 1. METHOD: A cross-sectional, semistructured telephone interview study was undertaken during spring 1997, with open-ended questions about Semester 1 experience of the four main themes. Qualitative data were analysed inductively and iteratively for emerging patterns and instances. RESULTS: All 34 tutors responded. Of these, 26/34 (76%) were male and 23 (68%) were medically qualified. Towards the end of each approximately (median) 20-minute interview, reflecting on the curriculum themes, tutors mostly identified with the basic/clinical science theme (Structure and Function). Almost half articulated a clear division (implicitly or explicitly 'fact' versus 'non-fact') between it and the 'other three' themes of behavioural science, population science, and ethicolegal aspects of professional practice, respectively. Only 14/34 (41%) of tutors (including both public health doctors) outlined the public health-based theme adequately without disclosing confusion, antagonism/indifference or difficulties/uncertainty. CONCLUSION: This study provides baseline qualitative insights about new PBL tutors' insecurities when facilitating student discussion across integrated content. Given the difficulties of recruiting suitable educators into this role and potential resource limitations, staff retention and development strategies still must confront the reality of PBL tutors' bemusement when they are placed outwith their usual comfort zones.     

Setting up a clinical skills learning facility

Objective  This paper outlines the considerations to be made when establishing a clinical skills learning facility.
Considerations  Establishing a clinical skills learning facility is a complex project with many possible options to be considered. A number of professional groups, undergraduate or postgraduate, may be users. Their collaboration can have benefits for funding, uses and promotion of interprofessional education. Best evidence and educational theory should underpin teaching and learning. The physical environment should be flexible to allow a range of clinical settings to be simulated and to facilitate a range of teaching and learning methods, supported by computing and audio-visual resources. Facilities should be available to encourage self-directed learning. The skills programme should be designed to support the intended learning outcomes and be integrated within the overall curriculum, including within the assessment strategy. Teaching staff may be configured in a number of ways and may be drawn from a variety of backgrounds. Appropriate staff development will be required to ensure consistency and quality of teaching with monitoring and evaluation to assure appropriate standards. Patients can also play a role, not only as passive teaching material, but also as teachers and assessors. Clinical, diagnostic and therapeutic equipment will be required, as will models and manikins. The latter will vary from simple part task trainers to highly sophisticated human patient simulators. Care must be taken when choosing equipment to ensure it matches specified requirements for teaching and learning.
Conclusion  Detailed planning is required across a number of domains when setting up a clinical skills learning facility.     

Evaluation of a handheld clinical decision support tool for evidence-based learning and practice in medical undergraduates

INTRODUCTION: Handheld computers (PDAs) uploaded with clinical decision support software (CDSS) have the potential to facilitate the adoption of evidence-based medicine (EBM) at the point-of-care among undergraduate medical students. Further evaluation of the usefulness and acceptability of these tools is required. METHODS: All 169 Year 4 undergraduate medical students at the University of Hong Kong completed a post-randomised controlled trial survey. Primary outcome measures were CDSS/PDA usefulness, satisfaction, functionality and utilisation. Focus groups were also conducted to derive complementary qualitative data on the students' attitudes towards using such new technology. RESULTS: Overall, the students found the CDSS/PDA useful (mean score = 3.90 out of 6, 95% confidence interval (CI) = 3.78, 4.03). They were less satisfied with the functional features of the CDSS (mean score = 3.45, 95% CI = 3.32, 3.59) and the PDA (mean score = 3.51 95% CI = 3.40, 3.62). Utilisation was low, with the average frequency of use less than once per week. Although students reported a need for information in patient care at least once daily, they infrequently used the CDSS in a clinical setting (20.4 +/- 10.4% of the time), with an average information retrieval success rate of 37.6 +/- 22.1% requiring 63.7 +/- 86.1 seconds. Multivariable regression shows that higher perceived CDSS/PDA usefulness was associated with more supportive faculty attitudes, greater knowledge of EBM, better computer literacy skills and increased use in a clinical setting. Greater satisfaction with the CDSS/PDA was associated with increased use in a clinical setting and higher successful search rates. Qualitative results were consistent with these quantitative findings and yielded additional information on students' underlying feelings that may explain the observations. CONCLUSIONS: While PDAs uploaded with the CDSS are able to provide students with better access to high quality information, improvements in faculty attitudes, students' knowledge of EBM and computer literacy skills, and having the CDSS specially designed for undergraduate use are essential to increasing student adoption of such point-of-care tools.