Effects of basic clinical skills training on objective structured clinical examination performance

OBJECTIVES: The aim of curriculum reform in medical education is to improve students' clinical and communication skills. However, there are contradicting results regarding the effectiveness of such reforms. METHODS: A study of internal medicine students was carried out using a static group design. The experimental group consisted of 77 students participating in 7 sessions of communication training, 7 sessions of skills-laboratory training and 7 sessions of bedside-teaching, each lasting 1.5 hours. The control group of 66 students from the traditional curriculum participated in equally as many sessions but was offered only bedside teaching. Students' cognitive and practical skills performance was assessed using Multiple Choice Question (MCQ) testing and an objective structured clinical examination (OSCE), delivered by examiners blind to group membership. RESULTS: The experimental group performed significantly better on the OSCE than did the control group (P < 0.01), whereas the groups did not differ on the MCQ test (P < 0.15). This indicates that specific training in communication and basic clinical skills enabled students to perform better in an OSCE, whereas its effects on knowledge did not differ from those of the traditional curriculum. CONCLUSION: Curriculum reform promoting communication and basic clinical skills are effective and lead to an improved performance in history taking and physical examination skills.     

Competency domains in an undergraduate Objective Structured Clinical Examination: their impact on compensatory standard setting

Context  Following a 15-week attachment in paediatrics and child health, general practice and dermatology medical students in their second clinical year at this medical school undertake a high-stakes assessment including an objective structured clinical examination (OSCE). There were 2 hypotheses. Firstly, groups of similar stations map to competency domains identifiable by factor analysis. Secondly, poor performance in individual domains is compensated for by achieving the required standard of performance across the whole assessment.
Methods  A total of 647 medical students were assessed by an OSCE during 5 individual examination sittings (diets) over 2 years. Ten scoring stations in the OSCE were analysed and confirmatory factor analysis performed comparing a 1-factor model (where all the stations are discrete entities related to one underlying domain) with a 3-factor model (where the stations load onto 3 domains from a previously reported exploratory factor analysis).
Results  The 3-factor model yielded a significantly better fit to the data (χ² = 15.3, P  <   0.01). Assessing the compensation data of 1 diet, 29 of 127 students failed in 1 or more domains described, whereas only 5 failed if compensation was allowed across all domains.
Discussion  Confirmatory factor analysis showed a significant fit of the data to previously described competency domains for a high-stakes undergraduate OSCE. Compensation within but not between competency domains would provide a more robust standard, improve validity, and substantially reduce the pass rate.     

Trialling a new way to learn clinical skills: systematic clinical appraisal and learning

AIM: To describe and evaluate the effectiveness of a new method of teaching clinical skills designed to increase students' active and self-directed learning as well as tutor feedback. METHODS: A total of 22 consenting Year 4 medical students undertaking general practice and general surgery clinical experience were involved in a pre- and post-test research design. In the initial period of the study, students were taught clinical skills in a traditional manner. In the second phase a clinical teaching strategy called systematic clinical appraisal and learning (SCAL) was utilised. This learning strategy involved active and self-directed learning, holistic care and immediate feedback. Students independently saw a patient and were asked to make judgements about the patient's potential diagnosis, tests required, management, psychosocial needs, preventive health requirements, and any ethical problems. These judgements were then compared with those of the clinical supervisor, who saw the same patient independently. Students recorded details for each consultation. Comparisons were made of the two study periods to examine whether the use of SCAL increased the number of students' independent judgements, perceived student learning, tutor feedback and self-directed learning. RESULTS: During the SCAL learning period, students reported making a greater number of statistically significant independent judgements, and receiving significantly increased tutor feedback in both general practice and general surgery. The number of learning goals set by students was not found to differ between the two periods in surgery but significantly increased in general practice in the SCAL period. Students' perceptions of their learning significantly increased in the SCAL period in surgery but not in general practice. During the traditional learning period in both settings, there was limited student decision-making about most aspects of care, but particularly those relating to prevention, psychosocial issues and ethics. CONCLUSIONS: The SCAL approach appears to offer some advantages over traditional clinical skills teaching. It appears to encourage active and independent decision-making, and to increase tutor feedback. Further exploration of the approach appears warranted.     

A web-based presentation of an undergraduate clinical skills curriculum

AIMS: The aim of this study was to use information and communications technology to present a curriculum of clinical skills in a user-friendly format. SETTING: A UK undergraduate medical school with a problem-based curriculum and a strong emphasis on proficiency in clinical skills. STUDY DESIGN: Case study describing the qualitative analysis of users' requirements and development of a web-based learning portfolio. EVALUATION: The study involved direct observation of users during a 'think-aloud' protocol, a validated software users' measurement inventory and a 17-item questionnaire designed to test whether 'SkillsBase' met its users' requirements. RESULTS: Students wanted a clear and flexible presentation of their skills curriculum that was easy to navigate, offered instructional material and standards for self- and peer assessment, offered useful Internet links, allowed them to compare their progress with school standards and peer norms, and could be used as a learning portfolio. During the think-aloud protocol, students made very few errors in data interpretation or navigation, and found SkillsBase easy to learn and aesthetically pleasing to use. They rated it higher on all measures of usability than standard commercial software. The questionnaire showed that it met most aspects of its design specification, although many students were doubtful that they would use its reflective function. It is available for inspection at http://www.skillsbase.man.ac.uk/. CONCLUSIONS: SkillsBase meets the design specification for a training and reflective aid to learning clinical skills and is very usable.     

Setting up a clinical skills learning facility

Objective  This paper outlines the considerations to be made when establishing a clinical skills learning facility.
Considerations  Establishing a clinical skills learning facility is a complex project with many possible options to be considered. A number of professional groups, undergraduate or postgraduate, may be users. Their collaboration can have benefits for funding, uses and promotion of interprofessional education. Best evidence and educational theory should underpin teaching and learning. The physical environment should be flexible to allow a range of clinical settings to be simulated and to facilitate a range of teaching and learning methods, supported by computing and audio-visual resources. Facilities should be available to encourage self-directed learning. The skills programme should be designed to support the intended learning outcomes and be integrated within the overall curriculum, including within the assessment strategy. Teaching staff may be configured in a number of ways and may be drawn from a variety of backgrounds. Appropriate staff development will be required to ensure consistency and quality of teaching with monitoring and evaluation to assure appropriate standards. Patients can also play a role, not only as passive teaching material, but also as teachers and assessors. Clinical, diagnostic and therapeutic equipment will be required, as will models and manikins. The latter will vary from simple part task trainers to highly sophisticated human patient simulators. Care must be taken when choosing equipment to ensure it matches specified requirements for teaching and learning.
Conclusion  Detailed planning is required across a number of domains when setting up a clinical skills learning facility.     

An analysis of standardised patient checklist errors and their effect on student scores

BACKGROUND: The frequency and nature of standardised patient (SP) recording errors during clinical performance examinations (CPX) have an effect on case scores and ultimately on pass/fail decisions. PURPOSE: To determine the effect of SP recording errors on case scores. METHODS: Standardised patients completed checklists immediately after each encounter. To determine checklist accuracy, multiple reviewers developed a checklist key for each student encounter studied. The total errors, the net errors, the errors of commission and omission and error rates by competency skill were analysed. RESULTS: The frequency of errors in history taking was greater than in physical examination, and the majority of errors were made in the students' favour. Summing the errors of commission and omission decreased the effect of total errors on student scores. CONCLUSIONS: High levels of SP recording accuracy are achievable. When errors occur, the net effect is usually in the students' favour.     

The impact of a new educational strategy on acquiring neonatology skills

OVERVIEW: A shortage of staff for teaching neonatology skills to large numbers of students, in small groups and following a new curriculum, necessitated an innovative educational strategy. This entailed the development and implementation of an interactive multimedia program (CD-ROM) to deliver information about skills and to demonstrate them. METHODS: Students had to study a specific skill using the CD-ROM and then practise in the Skills Laboratory, supported by lecturers who provided formative evaluation. OBJECTIVES: The aims of this study were to assess the students' perspectives on the new strategy, and to compare the skills of students following the new curriculum to those of students following the traditional curriculum, who do not follow structured programmes on practical skills but experience a practical neonatology rotation. RESULTS: The evaluation of the CD-ROM program was very favourable. The majority of students still preferred live demonstrations but found the CD-ROM useful for revision purposes. With the exception of one skill, endotracheal intubation, the new curriculum students were found to be as competent as the students following the traditional curriculum and performed mask ventilation and cardiac massage significantly better than them.     

Approaches to medical student remediation after a comprehensive clinical skills examination

OBJECTIVE: Most US medical schools conduct comprehensive clinical skills assessments during Years 3 and 4. This study explores strategies used to identify and remediate students who perform poorly on these assessments. METHODS: In the academic year 2005-06, we conducted 33 semi-structured interviews with individuals responsible for standard setting in and remediation after their schools' comprehensive clinical skills assessments. We coded interviews to identify major themes. RESULTS: Prior to remediation, some schools employed a 'verification' step to ensure the accuracy of the failing score or need for remediation. Participants described a remediation process that included some or all of 3 steps. Firstly, students' specific learning deficits were diagnosed. Next, students participated in remedial activities such as performance review sessions or practice with standardised or actual patients. Lastly, students were re-tested, usually with a shorter, more formative examination. All participants reported using a diagnostic step, most offered or required remedial activities and many re-tested, although schools varied in the emphasis placed on each step. Many participants cited the individualised attention students received from remediation faculty staff as a strength of their approach, although they raised concerns about the substantial time demands placed on remediation faculty. Most respondents reported some dissatisfaction with their school's remediation process, particularly uncertainty about efficacy or rigour. CONCLUSIONS: Schools vary in the intensity and scope of remediation offered to students who perform poorly on clinical skills assessments. Although many schools invest significant resources in remediation, the effect of these efforts on students' subsequent clinical performance is unknown.     

Promoting diagnostic problem representation

PURPOSE: Problem representation, as mediated by semantic qualifiers (SQs), has been associated with better diagnostic outcomes. The purpose of this study was to assess the effect of training medical students to use semantic abstractions as a means of building problem representations. METHODS: Sixty second-year medical students were assigned to either an intervention group (n = 20) or a control group (n = 40) during 8 months of an Essentials in Clinical Medicine course which used standardized patient-based workshops. Students were trained to transform findings into SQs and to use abstractions to compare and contrast diagnostic hypotheses. Students were assessed using a standardized patient data collection checklist, a post-encounter patient finding questionnaire (PFQ), and case summaries and write-ups. RESULTS: Experimental subjects used over twice as many SQs in their summaries as control group members (1.40 versus 0.63, P = 0.006). The correlation between checklist and PFQ scores was higher for the experimental group than for the control group (r = 0.70 versus r = 0.58, P 0.56). CONCLUSION: A short instructional intervention was successful in promoting the use of SQs and enabled students to recall elicited findings better. This intervention did not enhance data interpretation and diagnostic accuracy. Use of SQs may therefore be a necessary tool for efficient problem representation but one that is insufficient when used in isolation. The naturalistic setting used in this study imposed a number of limitations, implying that further research should test whether instructional efforts should also emphasize recognition of key patient findings and knowledge representation.     

Using standardised students in faculty development workshops to improve clinical teaching skills

PURPOSE: We describe the use of standardised students (SSs) in interdisciplinary faculty development programmes to improve clinical teaching skills. Standardised students are actual health professions students who are trained to portray a prototypical teaching challenge consistently across many encounters with different faculty participants. METHODS: The faculty development programmes described focused on the skills of providing feedback and brief clinical teaching. At the beginning of each session, each participant was videotaped in encounters with 2 different SSs. Using microteaching (an instructional method in which learners view short segments of their own videotaped performance and discuss the tapes with a facilitator, consultant or other workshop participants), each group of participants and instructors reviewed the tapes and reflected on the encounters, providing immediate feedback to participants and modelling different approaches to the same teaching problem. The same process was repeated with more complicated scenarios after 2 weeks and again after 6 months offering reinforcement, further practice and more sophisticated development of the strategies learned. Participants completed post-session evaluations and a follow-up telephone survey. RESULTS: A total of 36 faculty members from the colleges of medicine, dentistry, pharmacy and nursing participated in workshops in 2000-01. The workshops were rated as highly relevant to participants' teaching, and most participants reported that they had learned a great deal. Participants most appreciated reviewing the videotaped interactions, the feedback they received, the interactions with their colleagues, the interdisciplinary nature of the groups and the practical focus of the workshops. CONCLUSIONS: Standardised students provide a high fidelity, low risk, simulated environment in which faculty can reflect on and experiment with new teaching behaviours. Such encounters can enhance the effectiveness and impact of faculty development programmes to improve clinical teaching skills.     

Alligator attack: an illustration of the impact of early clinical exposure

CONTEXT AND PROCESS: The authors recount the importance of mentoring during medical education. A personal narrative of clinical exposure as a University of South Florida (USF) student at Tampa General Hospital illustrates the unparalleled value of unique clinical situations such as the emergency care of a patient following an alligator attack. The benefits of early clinical experience and integrated medical education to be obtained through a mentor relationship are explored. CONCLUSION: The clinical exposure of medical students is central to their development as clinicians.     

Effects of communication skills training on students'' diagnostic efficiency

This study evaluates the impact of a training programme in communications skills on subsequent diagnostic efficiency. Videotaped history-taking interviews conducted by groups of specially trained and control groups of students were rated for their diagnostic efficiency by two medical practitioners. Students in the trained group had shown greatly increased skills in interviewing and interpersonal effectiveness as a result of their training. A comparison of ratings given by the two experimentally naive, independent observers revealed that trained students were significantly better at eliciting full, relevant data from patients--they were diagnostically more efficient, but took no longer than their control group counterparts to elicit the information. Further research with the medical interview rating scale will clarify the skills required of medical students in interviewing and diagnosis and facilitate remedial training for students who show poor interview skills.     

The determination of the relevance of basic sciences learning objectives to clinical practice using a questionnaire survey

OBJECTIVES: Facilitating sufficient understanding of the basic sciences to underpin clinical practice is important in producing the good doctor. However, the inclusion of irrelevant material in the curriculum not only wastes valuable learning time, but may also hinder learning. The aim of this study was to determine how relevant staff and students thought respiratory basic science learning objectives were to medical practice. DESIGN: The study involved a survey using an anonymous questionnaire to determine whether the respiratory learning objectives stated in Year 1 were perceived as relevant to clinical practice. Each learning objective was rated as being 'relevant', 'not relevant' or of 'uncertain relevance'. SETTING: Dundee Medical School, UK. SUBJECTS: Junior and senior students and staff. RESULTS: Year 1 students considered the majority of the learning objectives to be relevant to clinical practice. Staff and senior students identified some respiratory learning objectives as not relevant to clinical practice, most of which were related to biochemistry. CONCLUSIONS: The identification of learning objectives with questionable relevance to clinical practice requires careful consideration to determine whether these should be removed from the course. Attention needs to be given to both the presentation and process by which material is delivered to students. Strategies to emphasise the clinical relevance of the basic sciences to students are discussed. Further research needs to be conducted to evaluate what knowledge is essential for producing good doctors.     

The impact of formal instruction in clinical examination skills on medical student performance -- the example of peripheral nervous system examination

BACKGROUND: Peripheral nervous system examination is an essential part of the full medical clerking of a patient. We have investigated the effectiveness of formal instruction in peripheral nervous system examination compared to the traditional bedside ward teaching that our students usually receive. METHOD: We instructed an unselected group of 22 medical students in peripheral nervous system examination in a clinical skills centre and evaluated them with a 12 item marking schedule before and after instruction. The performance of this group was then compared to the rest of their year (220 students) in an end of year OSCE, which included a neurology station assessing sensory examination of the lower limbs. RESULTS: Students formally instructed in neurology significantly improved their scores after instruction and scored 15% higher marks (90% vs. 75%) than the rest of their year in the end of year neurology OSCE station 2 months later (P < 0.01, Mann Whitney U-test). They did not perform significantly better in the OSCE overall. CONCLUSIONS: Formal instruction in neurological examination resulted in a significant increase in the end of year neurology OSCE station score compared to traditional heterogeneous teaching methods.     

Traditional finals and OSCEs in predicting consultant and self-reported clinical skills of PRHOs: a pilot study

INTRODUCTION: As we move from standard 'long case' final examinations to new objective structured formats, we need to ensure the new is at least as good as the old. Furthermore, knowledge of which examination format best predicts medical student progression and clinical skills development would be of value. METHODS: A group of medical students sat both the standard long case examination and the new objective structured clinical examination (OSCE) to introduce this latter examination to our Medical School for final MB. At the end of their pre-registration year, the group and their supervising consultants submitted performance evaluation questionnaires. RESULTS: Thirty medical students sat both examinations and 20 returned evaluation questionnaires. Of the 72 consultants approached, 60 (83%) returned completed questionnaires. No correlation existed between self- and consultant reported performance. The traditional finals examination was inversely associated with consultant assessment. Better performing students were not rated as better doctors. The OSCE (and its components) was more consistent and showed positive associations with consultant ratings across the board. DISCUSSION: Major discrepancies exist between the 2 examination formats, in data interpretation and practical skills, which are explicitly tested in OSCEs but less so in traditional finals. Standardised marking schemes may reduce examiner variability and discretion and weaken correlations across the 2 examinations. This pilot provides empirical evidence that OSCEs assess different clinical domains than do traditional finals. Additionally, OSCEs improve prediction of clinical performance as assessed by independent consultants. CONCLUSION: Traditional finals and OSCEs correlate poorly with one another. Objective structured clinical examinations appear to correlate well with consultant assessment at the end of the pre-registration house officer year.     

Evaluation of procedural skills training in an undergraduate curriculum

CONTEXT: A substantial proportion of medical students enter their intern year without any basic skills experience. Lack of experience is a significant source of stress for many junior doctors. OBJECTIVES: To evaluate the effect of a basic procedural skills tutorial for Year 3 medical students on their competence in relevant skills at Year 5. SUBJECTS: The control group consisted of 93 medical students who completed Year 3 in 1996. The intervention group consisted of 92 medical students who completed Year 3 in 1997. The intervention group received a practical skills tutorial in Year 3; the control group did not. Both groups were assessed on their practical skills competence during Year 5. METHODS: A 3-hour practical tutorial on injection and suturing techniques was delivered to the intervention group. The effectiveness of the intervention was assessed by self-reported experience of giving injections, inserting sutures and sustaining needlestick injuries, and by teacher-rated competency in four basic procedural skills. RESULTS: Students who received the Year 3 tutorial were significantly more likely to record a satisfactory assessment for their performance in all four basic skills compared with students who did not receive the tutorial. They were less likely than controls to refuse invitations to give injections, but not invitations to insert a suture, during Years 4 and 5. CONCLUSIONS: A single session of formalised teaching in procedural skills in the early stages of a medical degree can have long-term effectiveness in basic skills competence and may increase students' confidence to practise their skills.     

Peer-assisted learning: a novel approach to clinical skills learning for medical students

OBJECTIVE: This study aimed to determine whether peer-assisted learning (PAL) can enhance clinical examination skills training. METHODS: Three student trainers studied small-group theory and clinical examination and provided PAL as extra tuition for 86 trainees. Trainees watched an examination video, were videotaped practising the examination and, after constructive feedback, repeated the examination. Responses to PAL were evaluated to attain an overview of trainee and trainer performance using visual analogue and Likert scale analyses. Year-group review was undertaken using questionnaires. RESULTS: Trainees evaluated all aspects of PAL highly, including their post-training confidence in examination skills (mean > 7.7 on a 10-cm scale), indicating that the PAL was effective. Written comments confirmed the students perceived the sessions as well structured and of high quality. Compared with trainees in the first groups, those from later groups gave all parameters similar or higher gradings. Those for interest (P = 0.03) and appropriateness (P = 0.01) were significantly higher, suggesting that trainers may improve their technique with time. Students with previous degrees gave similar or lower gradings than standard entry students, with answers about post-training confidence and recommendation to friends being statistically lower (P < 0.006). Six months later, year-group analysis showed that 90% of trainees rated PAL highly, and 86% wished to become trainers. Of the trainers' year group, 79% perceived that PAL training could improve examination skills. CONCLUSIONS: In the context of clinical skills training, PAL was highly evaluated across many parameters, including confidence after training. Student interest and enthusiasm supports suggestions that PAL could be a useful adjunct to clinical skills training.     

Towards valid measures of self-directed clinical learning

AIM: To compare the validity of different measures of self-directed clinical learning. METHODS: We used a quasi-experimental study design. The measures were: (1) a 23-item quantitative instrument measuring satisfaction with the learning process and environment; (2) free text responses to 2 open questions about the quality of students' learning experiences; (3) a quantitative, self-report measure of real patient learning, and (4) objective structured clinical examination (OSCE) and progress test results. Thirty-three students attached to a single firm during 1 curriculum year in Phase 2 of a problem-based medical curriculum formed an experimental group. Thirty-one students attached to the same firm in the previous year served as historical controls and 33 students attached to other firms within the same module served as contemporary controls. After the historical control period, experimental group students were exposed to a complex curriculum intervention that set out to maximise appropriate real patient learning through increased use of the outpatient setting, briefing and supported, reflective debriefing. RESULTS: The quantitative satisfaction instrument was insensitive to the intervention. In contrast, the qualitative measure recorded a significantly increased number of positive statements about the appropriateness of real patient learning. Moreover, the quantitative self-report measure of real patient learning found high levels of appropriate learning activity. Regarding outpatient learning, the qualitative and quantitative real patient learning instruments were again concordant and changed in the expected direction, whereas the satisfaction measure did not. An incidental finding was that, despite all attempts to achieve horizontal integration through simultaneously providing community attachments and opening up the hospital for self-directed clinical learning, real patient learning was strongly bounded by the specialty interest of the hospital firm to which students were attached. Assessment results did not correlate with real patient learning. CONCLUSIONS: Both free text responses and students' quantitative self-reports of real patient learning were more valid than a satisfaction instrument. One explanation is that students had no benchmark against which to rate their satisfaction and curriculum change altered their tacit benchmarks. Perhaps the stronger emphasis on self-directed learning demanded more of students and dissatisfied those who were less self-directed. Results of objective, standardised assessments were not sensitive to the level of self-directed, real patient learning. Despite an integrated curriculum design that set out to override disciplinary boundaries, students' learning remained strongly influenced by the specialty of their hospital firm.     

Assessment of patient management skills and clinical skills of practising doctors using computer-based case simulations and standardised patients

CONTEXT: Standardised assessments of practising doctors are receiving growing support, but theoretical and logistical issues pose serious obstacles. OBJECTIVES: To obtain reference performance levels from experienced doctors on computer-based case simulation (CCS) and standardised patient-based (SP) methods, and to evaluate the utility of these methods in diagnostic assessment. SETTING AND PARTICIPANTS: The study was carried out at a military tertiary care facility and involved 54 residents and credentialed staff from the emergency medicine, general surgery and internal medicine departments. MAIN OUTCOME MEASURES: Doctors completed 8 CCS and 8 SP cases targeted at doctors entering the profession. Standardised patient performances were compared to archived Year 4 medical student data. RESULTS: While staff doctors and residents performed well on both CCS and SP cases, a wide range of scores was exhibited on all cases. There were no significant differences between the scores of participants from differing specialties or of varying experience. Among participants who completed both CCS and SP testing (n = 44), a moderate positive correlation between CCS and SP checklist scores was observed. There was a negative correlation between doctor experience and SP checklist scores. Whereas the time students spent with SPs varied little with clinical task, doctors appeared to spend more time on communication/counselling cases than on cases involving acute/chronic medical problems. CONCLUSION: Computer-based case simulations and standardised patient-based assessments may be useful as part of a multimodal programme to evaluate practising doctors. Additional study is needed on SP standard setting and scoring methods. Establishing empirical likelihoods for a range of performances on assessments of this character should receive priority.