Clinical skills in junior medical officers: a comparison of self-reported confidence and observed competence

BACKGROUND: The intern year is a key time for the acquisition of clinical skills, both procedural and cognitive. We have previously described self-reported confidence and experience for a number of clinical skills, finding high levels of confidence among Australian junior doctors. This has never been correlated with an objective measure of competence. AIMS AND HYPOTHESIS: We aimed to determine the relationship between self-reported confidence and observed competence for a number of routine, procedural clinical skills. METHODS: A group of 30 junior medical officers in their first postgraduate year (PGY1) was studied. All subjects completed a questionnaire concerning their confidence and experience in the performance of clinical skills. A competency-based assessment instrument concerning 7 common, practical, clinical skills was developed, piloted and refined. All 30 PGY1s then completed an assessment using this instrument. Comparisons were then made between the PGY1s' self-reported levels of confidence and tutors' assessments of their competence. RESULTS: A broad range of competence levels was revealed by the clinical skills assessments. There was no correlation between the PGY1s' self-ratings of confidence and their measured competencies. CONCLUSIONS: Junior medical officers in PGY1 demonstrate a broad range of competence levels for several common, practical, clinical skills, with some performing at an inadequate level. There is no relationship between their self-reported level of confidence and their formally assessed performance. This observation raises important caveats about the use of self-assessment in this group.     

Pre-registration house officers and ward-based learning: a 'new apprenticeship' model

INTRODUCTION: The pre-registration house officer (PRHO) year can be seen as a formal apprenticeship into the profession of medicine, and as central to the identity construction of the doctor. The year characteristically involves rotation between specialties, including attachment to ward-based 'firms', where consultants teach PRHOs. DISCUSSION: Teaching and learning in ward-based environments is under-researched, and the current literature displays a bias towards a psychological model of pedagogy that focuses upon transmission of knowledge and skills from one individual to another. Such a model offers a necessary, but not sufficient, explanation of how work-based learning occurs. Understanding the PRHO apprenticeship year should include reference to cultural dimensions to learning, especially socialisation into the profession. This constitutes an 'extended' (or 'hidden') curriculum model that may be theorised through contemporary ideas of activity learning within a 'new apprenticeship' framework. CONCLUSION: The dominant psychological model can lead to an expectation for a uniform method of teaching and learning in ward round contexts that (a) ignores important differences in educational climate between established communities of practice, and (b) orients both teachers and learners to one-to-one transmission and reception, rather than sensitising to how knowledge may be held across members of a working group. The latter shifts emphasis away from reception to issues of active access. PRHOs, as novices, are not relegated to passive learning roles, but may actively co-construct knowledge with experts, offering potential transformation of the practices of ward groups.     

Attitudes of medical students to old people: a cross-national comparative study

Medical students' attitudes to the elderly were compared at the start and finish of a 5-week clinical attachment in health care of the elderly at the Christchurch School of Medicine. The study investigated students in their first clinical year (fourth year of their medical course) over five terms using a questionnaire employing a Rosencranz-McNevin semantic differential scale to measure general attitudes to old age and a Likert scale to measure attitudes to medical care. A question was also asked about career preferences. There was significant improvement in attitudes measured by both scales (Rosencranz-McNevin P less than 0.001, Likert P less than 0.001). Students also showed an increase in interest in health care of the elderly as a career choice. When compared with two cohorts of students from Nottingham Medical School, attitudes were significantly better in the Christchurch group at the commencement of the run. Students at both schools showed an improvement in knowledge but this was more marked for Christchurch students.     

A survey of clinical teaching fellowships in UK medical schools

CONTEXT: Undergraduate medical education in the UK has changed considerably over the last decade. One development has involved the creation of teaching-specific posts for junior doctors by medical schools. These posts are generally termed 'clinical teaching fellowships', but it is not known how many of them exist, or whether they are similar in terms of educational activities, professional development, and research and clinical experience opportunities. METHODS: Teaching deans in all UK medical schools were sent a questionnaire relating to clinical teaching fellowships, and were asked to distribute a second set of different questionnaires to their clinical teaching fellows, which were to be returned to the authors separately. RESULTS: A total of 28 deans and 46 fellows responded. Fifteen medical schools had clinical teaching fellows and there appeared to be a total of 77 such posts in the UK. There was little uniformity in the activities undertaken within the posts. Deans who employed clinical teaching fellows were unanimously positive regarding the posts. Fellows were generally positive but expressed reservations relating to approval for postgraduate training, career development, deterioration in clinical skills, financial disincentives, credibility within one's own specialty, and provision of training and support. CONCLUSIONS: Clinical teaching fellow posts are generally enjoyed by fellows and valued by deans. Fellows carry out differing duties and their training in medical education is variable. The posts can be unstructured and may lack credibility to doctors outside medical education. Providing specific structured training in medical education, recognised at a national level, would help deal with these concerns.     

Evaluation of self-directed clinical education: validation of an instrument

AIM: To explore the evaluation of self-directed, integrated clinical education. METHODS: We delivered a quantitative and qualitative, self-report questionnaire to students through their web-based learning management system. The questionnaire was distributed 4 times over 1 year, each time in 2 parts. A generic part evaluated boundary conditions for learning, teaching activities and "real patient learning". Factor analysis with varimax rotation was used to validate the constructs that made up the scale and to stimulate hypotheses about how they interrelated. A module-specific part evaluated real patient learning of the subject matter in the curriculum. RESULTS: A total of 101 students gave 380 of a possible 404 responses (94%). The generic data loaded onto 4 factors, corresponding to: firm quality; hospital-based teaching and learning; community and out-patient learning, and problem-based learning (PBL). A 5-item quality index had content, construct and criterion validity. Quality differed greatly between firms. Self-evaluation of module-specific, real patient learning was also valid. It was strongly influenced by the specialty interests of hospital firms. CONCLUSIONS: Quality is a multidimensional construct. Self-report evaluation of real patient learning is feasible, and could be capitalised on to promote reflective self-direction. The social and material context of learning is an important dimension of educational quality.     

What do students actually do on an internal medicine clerkship? A log diary study

BACKGROUND: There are limited data on the amount of time students spend on teaching and learning while on internal medicine clerkships, and existing data suggest a wide international variation. Community-based teaching of internal medicine is now widespread; but its strengths and weaknesses compared to traditional hospital based teaching are still unclear. AIM: To determine the proportion of time students spend on different activities on an internal medicine clerkship, and to determine whether this differs in general practice and in hospital. In addition we aimed to determine students' views on the educational value and enjoyment of various activities. METHODS: Prospective completion of log diaries recording student activities. Each student was asked to complete the diary for two separate weeks of their internal medicine clerkship: one week of general practice-based teaching and one week of hospital-based teaching. RESULTS: The response rate was 68% (88/130). Students spent approximately 5.5 h per day on teaching and learning activities in both environments, with more time (50 min vs. 30 min, P = 0.007) on unsupervised interaction with patients in hospital than in general practice, and more time (53 min vs. 21 min, P < 0.001) undergoingassessment in general practice than in hospital. Standard deviations were wide, demonstrating the heterogeneous nature of the data. Students perceived supervised interaction with patients and teaching by doctors as the most educational activities in both environments, but found it even more educationally valuable and enjoyable in general practice than in hospital (mean score for educational value: 4.27 in general practice, 3.88 in hospital, P = 0.048; mean score for enjoyment 4.13 in general practice, 3.66 in hospital, P = 0.03). CONCLUSIONS: Students greatly value interactions with patients, perceiving these as both educational and enjoyable. Curriculum planners must continue to place patient-based learning at the centre of undergraduate medical education. The heterogeneity of the data suggests that individual students have very different experiences, despite apparently similar timetables.     

The impact of case specificity and generalisable skills on clinical performance: a correlated traits-correlated methods approach

Context  The finding of case or content specificity in medical problem solving moved the focus of research away from generalisable skills towards the importance of content knowledge. However, controversy about the content dependency of clinical performance and the generalisability of skills remains.
Objectives  This study aimed to explore the relative impact of both perspectives (case specificity and generalisable skills) on different components (history taking, physical examination, communication) of clinical performance within and across cases.
Methods  Data from a clinical performance examination (CPX) taken by 350 Year 3 students were used in a correlated traits−correlated methods (CTCM) approach using confirmatory factor analysis, whereby 'traits' refers to generalisable skills and 'methods' to individual cases. The baseline CTCM model was analysed and compared with four nested models using structural equation modelling techniques. The CPX consisted of three skills components and five cases.
Results  Comparison of the four different models with the least-restricted baseline CTCM model revealed that a model with uncorrelated generalisable skills factors and correlated case-specific knowledge factors represented the data best. The generalisable processes found in history taking, physical examination and communication were responsible for half the explained variance, in comparison with the variance related to case specificity.
Conclusions  Pure knowledge-based and pure skill-based perspectives on clinical performance both seem too one-dimensional and new evidence supports the idea that a substantial amount of variance contributes to both aspects of performance. It could be concluded that generalisable skills and specialised knowledge go hand in hand: both are essential aspects of clinical performance.     

The determination of the relevance of basic sciences learning objectives to clinical practice using a questionnaire survey

OBJECTIVES: Facilitating sufficient understanding of the basic sciences to underpin clinical practice is important in producing the good doctor. However, the inclusion of irrelevant material in the curriculum not only wastes valuable learning time, but may also hinder learning. The aim of this study was to determine how relevant staff and students thought respiratory basic science learning objectives were to medical practice. DESIGN: The study involved a survey using an anonymous questionnaire to determine whether the respiratory learning objectives stated in Year 1 were perceived as relevant to clinical practice. Each learning objective was rated as being 'relevant', 'not relevant' or of 'uncertain relevance'. SETTING: Dundee Medical School, UK. SUBJECTS: Junior and senior students and staff. RESULTS: Year 1 students considered the majority of the learning objectives to be relevant to clinical practice. Staff and senior students identified some respiratory learning objectives as not relevant to clinical practice, most of which were related to biochemistry. CONCLUSIONS: The identification of learning objectives with questionable relevance to clinical practice requires careful consideration to determine whether these should be removed from the course. Attention needs to be given to both the presentation and process by which material is delivered to students. Strategies to emphasise the clinical relevance of the basic sciences to students are discussed. Further research needs to be conducted to evaluate what knowledge is essential for producing good doctors.     

The medical school as a social organization: the sources of resistance to change

Medical schools vary by nation and by culture but, for students, the experience appears to be very similar. Also, despite a half-century of radical changes in medical practice, education as a process of socialization for the profession is relatively unchanged. At the same time, medical educators have frequently instituted curricular reforms. To analyse this history of reform without change, this paper first establishes what the content and structure of medical education is, and how it came to be that way; second it traces a process whereby the scientific mission of academic medicine has crowded out its social responsibility to train for society's most basic health-care delivery needs. The main argument is that medical education's manifest humanistic mission is little more than a screen for the research mission that is the major thrust of the institution's social structure.     

Teaching sociology to medical students

A recent change in many medical curricula has been the introduction of courses in the behavioural sciences. These courses, while introduced with the intention of emphasizing interpersonal and behavioural skills, have not been shown to lead to any of the changes towards which they are directed. Rather, there is evidence that students find these courses 'waffly' and boring. If the sociology component of these courses is to lead to change, then there is the need for a continuing process of evaluation and modification. In this paper we report upon one medical sociology course, its evaluation, subsequent modification and re-evaluation. Our evidence would suggest that sociology courses can lead to changed attitudes and values, but that such changes are contingent upon the overt application of relevant sociological concepts to the health care field.     

The impact of three years' ethics teaching,in an integrated medical curriculum,on students' proposed behaviour on meeting ethical dilemmas

OBJECTIVE: To evaluate the impact of 3 years' ethics teaching (30 hours mainly small group teaching in year 1, 14 hours mainly lecture-type teaching in years 2 and 3) on students' proposed behaviour on encountering ethical dilemmas. DESIGN: Cohort design. SETTING: University of Glasgow Medical School. SUBJECTS: A cohort of 111 students entering Glasgow University's new learner-centred, integrated medical curriculum; where ethics learning is formally assessed in years 1 and 5 only; in October 1996. MAIN OUTCOME MEASURE: Student answers consistent with consensus professional judgement on the ethical dilemmas posed by the vignettes of the Ethics and Health Care Survey Instrument. RESULTS: The instrument was completed pre- and post- year 1 and post- year 3 by 77%(85) of the cohort. There is a significant increase in the number of consensus answers given following the first year of the curriculum, but no further improvement was found. The odds ratio for giving the consensus answer post- year 1 relative to pre- year 1 was 1.42, 95% Confidence Interval (1.19, 1.71), P = 0.0001. Comparing post- year 3 to pre- year 1, odds ratio 1.30 (1.08, 1.57), P = 0.0062. Post- year 3 compared to post- year 1, odds ratio 0.91 (0.76, 1.10), P = 0.34. CONCLUSIONS: While small group ethics teaching can be effective in developing students' normative identification with the profession of medicine, its effectiveness is dependent on the amount of small group teaching provided. The lack of formal assessment in years 2 and 3 is also felt to contribute to the lack of impact. This information will inform future curriculum development.     

Geriatric medicine: does teaching alter medical students' attitudes to elderly people?

Summary. The development of a questionnaire to assess the attitudes of medical students towards old people is described. Principal components analysis of the responses of 114 first-year medical students revealed two orthogonal factors, named negative attitudes and medical intervention. Scores on these factors were compared among three groups of medical students: first-year students, 64 clinical phase medical students prior to a geriatric medicine course, and 69 medical students who had completed a geriatric medicine course. Negative attitudes scores did not differ between first year and the clinical years, but were reduced after the geriatric medicine course. Scores on the medical intervention factor reduced significantly from first year to the clinical years and were not reduced further by the geriatric medicine course. Women tended to have lower scores on negative attitudes. Medical students appeared to change their attitudes concerning the degree to which medical intervention is appropriate as a result of preclinical or general medical experience. However, their reservations concerning the reward to be gained from working with elderly people were stable over the same periods, but were altered by a course in geriatric medicine.     

Putting your course on the Web: lessons from a case study and systematic literature review

BACKGROUND: Education via the Internet offers enormous potential, but many online courses are pedagogically or technically weak and many good projects are never mainstreamed. METHOD: In drawing up our recommendations to address the issues around putting a course on the web, we drew on 3 main sources of data: an extensive in-depth course evaluation; a systematic review of the literature, and questions raised by participants on our training-the-trainers courses. RECOMMENDATIONS: For any web-based course to succeed, 10 overlapping and iterative areas of activity must be addressed. These are: the market for the course; course aims and intended learning outcomes; choice of software platform; staff training needs; writing high quality study materials; design features for active learning; technical and administrative challenges; evaluation and quality improvement; mainstreaming the course within the institution, and financial viability.     

Testing learning theories: the NUL hypothesis

INTRODUCTION: In the spirit of Scrooge and the Grinch, I describe the nasty unpalatable learning (NUL) hypothesis as a new theory of learning. This is an exercise to explore how difficult it would be to develop a new learning theory by reversing the social and altruistic tenets of most existing learning theories. THE NUL HYPOTHESIS: The NUL hypothesis has four propositions. These are: (1) the value of rote learning; (2) the usefulness of authority figures in teaching within a clear hierarchy; (3) the use of fear in teaching, and (4) the value of internal consistency without any necessary connection to the outside world. CONCLUSIONS: The NUL hypothesis may promote reflection not only on the construction of learning theories, but also on the design of critical experiments. It also seems to imply that efficiency may be a better measure of learning success than effectiveness.     

Assessing the relationship between peer and facilitator evaluations in case-based learning

OBJECTIVES: Attempts to validate peer evaluation and to incorporate it into the curriculum have met with mixed results. The purpose of this study was to assess the use of peer evaluations in a Year 1 case-based learning course. METHODS: As part of the formal grading process for the course, all faculty facilitators (n = 69 over 3 years) completed a 12-item evaluation form for each student at the conclusion of each case. As part of a course assignment, students (n = 415 over 3 years) completed brief evaluations of their peers based on 2 criteria: the overall quality of written reports, and participation in group discussion. In addition, students provided anonymous feedback in the written end-of-course evaluation about the peer evaluation process, and faculty were asked to comment during the wrap-up luncheon for small-group facilitators. RESULTS: Response rates for the 3 Year 1 medical student classes ranged from 95% to 99%. The average number of peer evaluations completed for each student was 4.6. The G coefficients for the rater-nested-within-person generalisability study were 0.52 for written reports and 0.60 for group participation; both were based on an average of 4-5 ratings. Correlation coefficients between peer and faculty evaluations in each of the 3 consecutive years of the course ranged from 0.46 to 0.63; all were statistically significant at P < 0.001. A correction for attenuation suggests that the true score correlation between faculty and peer measures is near 1.0. DISCUSSION: This study provides strong evidence that facilitator and peer ratings measure similar constructs and shows that, even among Year 1 medical students, peer evaluation can be conducted in a valid manner.     

Classical Greek and Roman rhetoric and the modern audience

The formal structuring of oral discourse or rhetoric was highly developed in antiquity. Both Greek and Roman authorities on the subject codified for orators an arrangement of material and a contextual format which have utility in the present day. The art of public lecturing should encompass relevance of material, structure of presentation and style of delivery in order to render the whole enjoyable and memorable. Teaching does not cause learning, but skilful rhetorical technique can imbue the student with a potent desire for further self-directed study. In this field, the ancient is auxiliary to the modern.     

The assessment of an examination in pathology consisting of multiple-choice, practical and short essay questions

The rationale for assessing medical students' achievement in pathology with examinations consisting of multiple-choice, practical and short essay items is provided. In addition, these three-part examinations are assessed with reference to their reliabilities and their content, criterion and construct validities.     

Integrating human factors into the medical curriculum

Background  The study of human factors is a scientific discipline that deals with the interactions between human beings and the elements of a system. This is important because shortcomings in these areas, if unchecked, can result in adverse outcomes. Research into human factors in industries where safety is paramount has provided the basis of countermeasures against human error. Adverse outcomes in medicine resulting from human error exact a high cost in both patient suffering and financial outlay.
CRM training  One of the approaches used to minimise the effect of human error is to train people in a set of knowledge, skills and attitudes that underpin the domain-specific competencies for that profession. These are referred to as non-technical skills (NTS). In aviation, such an approach has been shown to be both translatable from the training environment to the workplace and effective in reducing adverse outcomes.
Discussion  Medicine has incorporated this style of training, usually centred around simulator-based courses, but as yet in a piecemeal, episodic fashion which relies on participants volunteering to attend courses. Unlike other industries there is no systematic approach to linking the content of this teaching with the more conventional range of topics. As a consequence it is difficult to assess the impact of human factors training in medicine. This is partly because very little work has been done to date in identifying the key non-technical skills required in medicine, and the overall experience of workplace based assessment is limited. Lessons from other high reliability organisations may help to address the main challenges of developing the content, integrating it into the curriculum, reinforcing the concepts in the workplace through staff development and establishing its role in summative assessment.     

Professionalism and the basic sciences: an untapped resource

CONTEXT: Medical, technological and societal developments influence doctors' professional responsibilities and present challenges to educating medical students about professionalism. Medical education about professionalism generally focuses on behaviours and competencies which are taught primarily by clinicians in clinical courses and settings. DISCUSSION: Many professional competencies in medicine parallel those in science. We consider here whether medical professionalism can also be taught through the basic science courses which often initiate medical education, and which are typically taught by scientists. CONCLUSIONS: Like doctors, basic science faculty staff can teach professional competencies to medical students. Science faculty are well situated to teach professional competencies and should do so. They can model how to pursue evidence and manage conflicting information. They can also provide explicit messages to students about professional competencies and their value, and create learning objectives that reinforce those messages.