Development of self-assessment skills in medical students

The purpose of this study was to examine the ability of third-year medical students to evaluate accurately their own level of knowledge and performance of skills associated with anaesthesia. A total of 130 students participated in the project by completing a self-assessment form which was later matched with the corresponding instructors' evaluation. Analysis of the data identified only 4.6% of the students' self-assessments to be in general agreement with those evaluations prepared by their instructors. When the data was divided according to the placement of the anaesthesia clerkship within the third-year curriculum no significant trends of any type were shown. Students did not improve in the course of the year in their ability to evaluate themselves accurately according to the objectives established by their clinical instructors. The nature of the evaluations prepared by the clinical instructors remained fairly constant, indicating that their objectives and criteria for acceptable performance remained unchanged throughout the year.     

Medical students' perceptions of identity in communication skills training: a qualitative study

Objectives  Communication skills training in undergraduate medical education is considered to play an important role in medical students' formation of their professional identity. This qualitative study explores Year 1 students' perceptions of their identities when practising communication skills with real patients.
Methods  A total of 23 individual semi-structured interviews and two focus group discussions were conducted with 10 students during their first year of communication skills training. All interviews and discussions were audio-recorded, transcribed and analysed for emergent themes relating to identity.
Results  Students struggled to communicate professionally with patients because of a lack of clinical knowledge and skills. Consequently, students enacted other identities, yet patients perceived them differently, causing conversational ambiguities.
Discussion  Students' perceptions challenge educational goals, suggesting that there is limited potential for the formation of professional identity through early training. Teacher-doctors must acknowledge how students' low levels of clinical competence and patients' behaviour complicate students' identity formation.     

Not knowing that they do not know: self-assessment accuracy of third-year medical students

PURPOSE: The development of self-regulated learning is a major focus of our problem-based learning (PBL) medical programme. Students who are unsuccessful in assessments often seem to lack insight into the standard of their own performance, yet the ability to self-assess accurately is essential for the effective self-management of learning. The aim of this project was to evaluate the accuracy of self- and peer-assessment according to academic performance. METHOD: In 2004, 175 3rd-year students undertook an integrated, case-based, short-essay, formative assessment. After the assessment they were provided with model answers and marking criteria. Students marked their own assessment paper and the paper of one of their peers. Assessment papers were subsequently marked by faculty members. The following data was available for each student: self-mark, faculty-mark, score awarded by a peer and the score that they awarded to their peer. Self-assessment and peer-assessment ability was compared to overall academic performance. RESULTS: Low-achieving students score themselves and their peers generously. High-achieving students score themselves more harshly than faculty. However, they score their peers accurately. CONCLUSION: In the 3rd year of the programme low-achieving students are unable to assess accurately the quality of their own work or the work of their peers in a formative written assessment. The PBL curriculum does not guarantee the appropriate development of self-assessment skills.     

Evaluation of procedural skills training in an undergraduate curriculum

CONTEXT: A substantial proportion of medical students enter their intern year without any basic skills experience. Lack of experience is a significant source of stress for many junior doctors. OBJECTIVES: To evaluate the effect of a basic procedural skills tutorial for Year 3 medical students on their competence in relevant skills at Year 5. SUBJECTS: The control group consisted of 93 medical students who completed Year 3 in 1996. The intervention group consisted of 92 medical students who completed Year 3 in 1997. The intervention group received a practical skills tutorial in Year 3; the control group did not. Both groups were assessed on their practical skills competence during Year 5. METHODS: A 3-hour practical tutorial on injection and suturing techniques was delivered to the intervention group. The effectiveness of the intervention was assessed by self-reported experience of giving injections, inserting sutures and sustaining needlestick injuries, and by teacher-rated competency in four basic procedural skills. RESULTS: Students who received the Year 3 tutorial were significantly more likely to record a satisfactory assessment for their performance in all four basic skills compared with students who did not receive the tutorial. They were less likely than controls to refuse invitations to give injections, but not invitations to insert a suture, during Years 4 and 5. CONCLUSIONS: A single session of formalised teaching in procedural skills in the early stages of a medical degree can have long-term effectiveness in basic skills competence and may increase students' confidence to practise their skills.     

Does feedback matter? Practice‐based learning for medical students after a multi‐institutional clinical performance examination

CONTEXT: Achieving competence in 'practice-based learning' implies that doctors can accurately self- assess their clinical skills to identify behaviours that need improvement. This study examines the impact of receiving feedback via performance benchmarks on medical students' self-assessment after a clinical performance examination (CPX). METHODS: The authors developed a practice-based learning exercise at 3 institutions following a required 8-station CPX for medical students at the end of Year 3. Standardised patients (SPs) scored students after each station using checklists developed by experts. Students assessed their own performance immediately after the CPX (Phase 1). One month later, students watched their videotaped performance and reassessed (Phase 2). Some students received performance benchmarks (their scores, plus normative class data) before the video review. Pearson's correlations between self-ratings and SP ratings were calculated for overall performance and specific skill areas (history taking, physical examination, doctor-patient communication) for Phase 1 and Phase 2. The 2 correlations were then compared for each student group (i.e. those who received and those who did not receive feedback). RESULTS: A total of 280 students completed both study phases. Mean CPX scores ranged from 51% to 71% of items correct overall and for each skill area. Phase 1 self-assessment correlated weakly with SP ratings of student performance (r = 0.01-0.16). Without feedback, Phase 2 correlations remained weak (r = 0.13-0.18; n = 109). With feedback, Phase 2 correlations improved significantly (r = 0.26-0.47; n = 171). Low-performing students showed the greatest improvement after receiving feedback. CONCLUSIONS: The accuracy of student self-assessment was poor after a CPX, but improved significantly with performance feedback (scores and benchmarks). Videotape review alone (without feedback) did not improve self-assessment accuracy. Practice-based learning exercises that incorporate feedback to medical students hold promise to improve self-assessment skills.     

Does ultrasound training boost Year 1 medical student competence and confidence when learning abdominal examination?

OBJECTIVES: Learning to perform physical examination of the abdomen is a challenge for medical students. Medical educators need to find engaging, effective tools to help students acquire competence and confidence in abdominal examination techniques. This study evaluates the added value of ultrasound training when Year 1 medical students learn abdominal examination. METHODS: The study used a randomised trial with a wait-list control condition. Year 1 medical students were randomised into 2 groups: those who were given immediate ultrasound training, and those for whom ultrasound training was delayed while they received standard instruction on abdominal examination. Standardised patients (SPs) used a clinical skills assessment (CSA) checklist to assess student abdominal examination competence on 2 occasions - CSA-1 and CSA-2 - separated by 8 weeks. Students also estimated SP liver size for comparison with gold-standard ultrasound measurements. Students completed skills confidence surveys. RESULTS: Proficiency in abdominal examination technique acquired from traditional instruction boosted with ultrasound training showed no advantage at CSA-1. However, at CSA-2 the delayed ultrasound training group showed significant improvement. Students uniformly underestimated SP liver sizes and the estimates were not affected by ultrasound training. Student confidence in both groups improved from baseline to CSA-1 and CSA-2. CONCLUSIONS: Ultrasound training as an adjunct to traditional means of teaching abdominal examination improves students' physical examination technique after students have acquired skills with basic examination manoeuvres.     

Validation of MSAT: an instrument to measure medical students' self-assessed confidence in musculoskeletal examination skills

CONTEXT: Self-assessment promotes reflective practice, helps students identify gaps in their learning and is used in curricular evaluations. Currently, there is a dearth of validated self-assessment tools in rheumatology. We present a new musculoskeletal self-assessment tool (MSAT) that allows students to assess their confidence in their skills in and knowledge of knee and shoulder examination. OBJECTIVES: We aimed to validate the 15-item MSAT, addressing its construct validity, internal consistency, responsiveness, repeatability and relationship with competence. METHODS: Participants were 241 Year 3 students in Newcastle upon Tyne and 113 Year 3 students at University College London, who were starting their musculoskeletal skills placement. Factor analysis explored the construct validity of the MSAT; Cronbach's alpha assessed its internal consistency; standardised response mean (SRM) evaluated its responsiveness, and test-retest, before and after a pathology lecture, assessed its repeatability. Its relationship with competence was explored by evaluating its correlation with shoulder and knee objective structured clinical examinations (OSCEs). Results The MSAT was valid in distinguishing the 5 domains it intended to measure: clinical examination of the knee; clinical examination of the shoulder; clinical anatomy of the knee and shoulder; history taking, and generic musculoskeletal anatomical and clinical terms. It was internally consistent (alpha = 0.93), responsive (SRM 0.6 in Newcastle and 2.2 in London) and repeatable (intraclass correlation coefficient 0.97). Correlations between MSAT scores and OSCE scores were weak (r < 0.2). CONCLUSIONS: The MSAT has strong psychometric properties, thereby offering a valid approach to evaluating the self-assessment of confidence in examination skills by students. Confidence does not necessarily reflect competence; future research should clarify what underpins confidence.     

A longitudinal study of self-assessment accuracy

AIM: Although studies have examined medical students' ability to self-assess their performance, there are few longitudinal studies that document the stability of self-assessment accuracy over time. This study compares actual and estimated examination performance for three classes during their first 3 years of medical school. METHODS: Students assessed their performance on classroom examinations and objective structured clinical examination (OSCE) stations. Each self-assessment was then contrasted with their actual performance using idiographic (within-subject) methods to define three measures of self-assessment accuracy: bias (arithmetic differences of actual and estimated scores), deviation (absolute differences of actual and estimated scores), and covariation (correlation of actual and estimated scores). These measures were computed for four intervals over the course of 3 years. Multivariate analyses of variance and correlational analyses were used to evaluate the stability of these measures. RESULTS: Self-assessment accuracy measures were relatively stable over the first 2 years of medical school with a decease occurring in the third year. However, the correlational analyses indicated that the stability of self-assessment accuracy was comparable to the stability of actual performance over this same period. CONCLUSION: The apparent decline in accuracy in the third year may reflect the transition from familiar classroom-based examinations to the substantially different clinical examination tasks of the third year OSCE. However, the stability of self-assessment accuracy compares favorably with the stability of actual performance over this period. These results suggest that self-assessment accuracy is a relatively stable individual characteristic that may be influenced by task familiarity.     

Clinical skills in junior medical officers: a comparison of self-reported confidence and observed competence

BACKGROUND: The intern year is a key time for the acquisition of clinical skills, both procedural and cognitive. We have previously described self-reported confidence and experience for a number of clinical skills, finding high levels of confidence among Australian junior doctors. This has never been correlated with an objective measure of competence. AIMS AND HYPOTHESIS: We aimed to determine the relationship between self-reported confidence and observed competence for a number of routine, procedural clinical skills. METHODS: A group of 30 junior medical officers in their first postgraduate year (PGY1) was studied. All subjects completed a questionnaire concerning their confidence and experience in the performance of clinical skills. A competency-based assessment instrument concerning 7 common, practical, clinical skills was developed, piloted and refined. All 30 PGY1s then completed an assessment using this instrument. Comparisons were then made between the PGY1s' self-reported levels of confidence and tutors' assessments of their competence. RESULTS: A broad range of competence levels was revealed by the clinical skills assessments. There was no correlation between the PGY1s' self-ratings of confidence and their measured competencies. CONCLUSIONS: Junior medical officers in PGY1 demonstrate a broad range of competence levels for several common, practical, clinical skills, with some performing at an inadequate level. There is no relationship between their self-reported level of confidence and their formally assessed performance. This observation raises important caveats about the use of self-assessment in this group.     

The effect of professional skills training on patient-centredness and confidence in communicating with patients

OBJECTIVES: The effect of introducing professional skills training on students' patient-centred attitudes and perceptions of ability to communicate was examined. The professional skills training included weekly training in communication skills, ethics and law, and clinical skills. METHODS: Consecutive cohorts of medical students receiving a traditional pre-clinical curriculum (n = 199) and a new curriculum including professional skills training (n = 255) were compared. Students completed the Doctor-Patient Scale to assess patient-centred attitudes and an 11-item scale to assess confidence in their ability to communicate with patients. Students completed the measures at the start of Year 1 and the end of Year 2. RESULTS: Students receiving the professional skills training showed increased confidence in communicating with patients and increases in 2 dimensions of patient-centredness ('holistic care' and 'patient decision making'). Students receiving the traditional curriculum showed increased nervousness in talking to patients. Gender and ethnic differences were found in patient-centredness and confidence in communicating, which were maintained over time. CONCLUSIONS: The introduction of professional skills training was successful in improving students' confidence in their ability to perform specific communicative behaviours and increasing patient-centredness relative to a traditional curriculum.     

Teaching brief motivational interviewing to Year three medical students

OBJECTIVES: In 2005, the authors developed and tested a curriculum to teach Year 3 Yale University medical students a behaviour change counselling approach called 'brief motivational interviewing' (BMI). Brief motivational interviewing is a patient-centred approach designed to promote changes in patient behaviour within the time constraints imposed by a busy medical practice. METHODS: Standardised patients/instructors delivered the curriculum within a single 2-hour training episode using a teaching acronym called 'CHANGE' to promote the students' learning. The authors used a pretest, post-test and 4-week follow-up design to assess students' BMI skills (as measured by the Helpful Response Questionnaire), knowledge and attitudes toward the approach. RESULTS: Students successfully increased their use of BMI-consistent behaviours, primarily by increasing the frequency and depth of their reflections and by reducing the frequency with which they incorporated communication roadblocks and closed questions into their responses (all P-values < or = 0.05). Students also showed increases in BMI knowledge, interest in the approach, confidence in their ability to use BMI, and commitment to incorporating BMI skills into their future medical practice (all P-values < or = 0.05). CONCLUSIONS: The findings suggest that Year 3 medical students can learn basic BMI skills and knowledge and develop positive attitudes toward the approach within a relatively short period of time. The authors discuss the study's limitations and future directions for teaching students BMI.     

Medical students' essay-writing skills: criteria-based self- and tutor-evaluation and the role of language background

OBJECTIVES: An exercise is described which aimed to make clear to first-year undergraduate medical students the expected writing skills required for an essay examination in one discipline. SUBJECTS: Many students were from a non-English speaking background and over one-third of students, regardless of language background, had limited experience in this type of essay writing. PROCEDURE: For this exercise, a practice essay was written by each student for formative assessment. The essay was rated by a tutor and by the student according to well-defined criteria. This allowed for comparisons to be made in a structured and objective way between the judgements of the student and the assessor. RESULTS: Students found the exercise to be very useful, although whether essay writing skills actually improved could not be established. Students from non-English speaking backgrounds tended to be most harsh in their self-evaluations, yet tutor-evaluations generally showed these students to have better writing skills than other students. Indeed, correlations between self- and tutor-evaluations were quite low. CONCLUSIONS: It is evident that students and their educators may be unclear about each others' expectations. By making explicit the requirements of an exercise, misunderstandings may be minimized and it is possible that student performance could improve, though further research is required to verify these hypotheses. It is suggested that students should be encouraged to evaluate their own work and should be instructed in writing skills throughout their medical degree education.     

Students' attitudes and potential behaviour with regard to whistle blowing as they pass through a modern medical curriculum

OBJECTIVE: To examine students' attitudes and potential behaviour with regard to whistle blowing as they progress through a modern undergraduate medical curriculum. DESIGN: Cohort design. SETTING: University of Glasgow Medical School. SUBJECTS: A cohort of students entering Glasgow University's new learner-centred, integrated medical curriculum in October 1996. METHODS: Students' pre- and post-Year 1, post-Year 3 and post-Year 5 responses to the whistle blowing vignette of the Ethics in Health Care Instrument (EHCI) were examined quantitatively and qualitatively. Analysis of students' multichoice answers enabled measurement of movement towards professional consensus opinion. Analysis of written justifications helped determine whether their reasoning was consistent with professional consensus and enabled measurement of change in knowledge content and recognition of the values inherent in the vignette. Themes in students' reasoning behind their decisions of whether or not to whistle blow were also identified. RESULTS: There was little improvement in students' performance as they progressed through the curriculum in terms of their proposed behaviour on meeting the whistle blowing scenario. There was also no improvement in the quality of justifications provided. Students' reasoning on whether or not to whistle blow was found to change as the curriculum progressed. CONCLUSIONS: The EHCI has the potential to elicit students' attitudes towards ethical issues at entry to medical school and to measure change as they progress through the curriculum. Students should be encouraged to contemplate dilemmas from all ethical standpoints and consider relevant legal implications. Whistle blowing should be addressed as part of the wider domain of professionalism.     

Effects of basic clinical skills training on objective structured clinical examination performance

OBJECTIVES: The aim of curriculum reform in medical education is to improve students' clinical and communication skills. However, there are contradicting results regarding the effectiveness of such reforms. METHODS: A study of internal medicine students was carried out using a static group design. The experimental group consisted of 77 students participating in 7 sessions of communication training, 7 sessions of skills-laboratory training and 7 sessions of bedside-teaching, each lasting 1.5 hours. The control group of 66 students from the traditional curriculum participated in equally as many sessions but was offered only bedside teaching. Students' cognitive and practical skills performance was assessed using Multiple Choice Question (MCQ) testing and an objective structured clinical examination (OSCE), delivered by examiners blind to group membership. RESULTS: The experimental group performed significantly better on the OSCE than did the control group (P < 0.01), whereas the groups did not differ on the MCQ test (P < 0.15). This indicates that specific training in communication and basic clinical skills enabled students to perform better in an OSCE, whereas its effects on knowledge did not differ from those of the traditional curriculum. CONCLUSION: Curriculum reform promoting communication and basic clinical skills are effective and lead to an improved performance in history taking and physical examination skills.     

Factors influencing the communication skills of first-year clinical medical students

The aim of this study was to examine the factors influencing medical students' communication skills. The sample comprised all first-year clinical students. Thirty-two received teaching in communication skills during the year; the remaining 56 did not. Students' career preferences, attitudes towards communication skills and confidence in their ability to communicate with patients were assessed by questionnaire at the beginning and end of the year. At the end of the year each student was videotaped interviewing a simulated patient. Students' communication skills were assessed on the basis of this interview by raters using a standardized rating scale, and by patient questionnaires. While there was some evidence that brief communication skills training improved skills, sex of student was a more significant predictor of level of skill. Students who perceived communication skills as less relevant to medicine and those who were more confident about their own communication skills were more likely to prefer a career in hospital medicine. Students' judgements of their ability to communicate effectively were poor. In the main there was no relationship between confidence and level of skill: where they were related, the association was negative. The benefits from communication skills training might be enhanced by involving hospital doctors in the teaching, and providing students with detailed video feedback on their skills at the outset.     

Trialling a new way to learn clinical skills: systematic clinical appraisal and learning

AIM: To describe and evaluate the effectiveness of a new method of teaching clinical skills designed to increase students' active and self-directed learning as well as tutor feedback. METHODS: A total of 22 consenting Year 4 medical students undertaking general practice and general surgery clinical experience were involved in a pre- and post-test research design. In the initial period of the study, students were taught clinical skills in a traditional manner. In the second phase a clinical teaching strategy called systematic clinical appraisal and learning (SCAL) was utilised. This learning strategy involved active and self-directed learning, holistic care and immediate feedback. Students independently saw a patient and were asked to make judgements about the patient's potential diagnosis, tests required, management, psychosocial needs, preventive health requirements, and any ethical problems. These judgements were then compared with those of the clinical supervisor, who saw the same patient independently. Students recorded details for each consultation. Comparisons were made of the two study periods to examine whether the use of SCAL increased the number of students' independent judgements, perceived student learning, tutor feedback and self-directed learning. RESULTS: During the SCAL learning period, students reported making a greater number of statistically significant independent judgements, and receiving significantly increased tutor feedback in both general practice and general surgery. The number of learning goals set by students was not found to differ between the two periods in surgery but significantly increased in general practice in the SCAL period. Students' perceptions of their learning significantly increased in the SCAL period in surgery but not in general practice. During the traditional learning period in both settings, there was limited student decision-making about most aspects of care, but particularly those relating to prevention, psychosocial issues and ethics. CONCLUSIONS: The SCAL approach appears to offer some advantages over traditional clinical skills teaching. It appears to encourage active and independent decision-making, and to increase tutor feedback. Further exploration of the approach appears warranted.     

Patients teach students: partners in arthritis education

CONTEXT: A large metropolitan teaching hospital within The Northern Clinical School, University of Sydney. OBJECTIVE: To assess whether students taught by trained patients (Patient Partners) acquire the same levels of competence in musculoskeletal examination skills for arthritis as students taught by Consultant Rheumatologists. SUBJECTS: Year four medical students in a six-year Undergraduate Medical Programme. METHOD: Students randomized to eight tutorial groups were taught musculoskeletal examination skills in a 75-90 minute tutorial. Four groups were taught by Consultants with an untrained patient present and four groups were taught by Patient Partners. RESULTS: Students' mean self-ratings of skill before and after their tutorial were summed. For both groups, self-ratings before the tutorial were similar. After the tutorial both groups showed substantial gains in levels of skill. Patient Partners' ratings of students' taught by either Consultants or Partners were comparable. CONCLUSIONS: Patient Partners are at least equal to Consultant Rheumatologists in the teaching of musculoskeletal examination techniques for arthritis.