The effect of an extended hospital induction on perceived confidence and assessed clinical skills of newly qualified pre-registration house officers

BACKGROUND: Recent studies raise concerns over the preparedness of newly qualified doctors for the role of the pre-registration house officer (PRHO). This study aimed to assess self-perception of preparedness, objective assessment of core clinical skills and the effect of an extended clinical induction programme prior to commencing full duties. METHODS: A group of 26 newly qualified doctors from 1 district general hospital underwent an extended 5-day, ward-based induction programme. The participants completed questionnaires on their own perceptions of their preparedness for PRHO duties and underwent an objective structured clinical examination (OSCE) of 4 core clinical skills prior to induction, on completion of induction and 1 month into working life. RESULTS: At the outset PRHOs had low perceptions of their own capabilities in all clinical scenarios and skills. Most perceptions improved after induction, although in 2 clinical areas they felt even less confident. One month into post there were significant improvements in all areas. Only 1 PRHO passed all 4 clinical skills assessments at the pre-induction assessment. Seven (26%) failed on 1 or more skills at the post-induction assessment. However, all participants were deemed competent in all skills at the 1-month assessment. CONCLUSION: Newly qualified doctors do not feel prepared for PRHO duties and objectively are not competent in basic clinical skills. An extended induction improves preparedness in some but not all clinical areas and improves performance of objectively assessed clinical skills.     

Improving communication: a practical programme for teaching trainees about communication issues in the general practice consultation

This paper describes a teaching programme, for use in general practice vocational training, which provides a theoretical and practical framework for exploring key aspects of the consultation with trainees. A particular emphasis is on the educational or 'cognitive' outcomes of the consultation and skills for improving them. The five stages of the programme are described and an example of experience of each stage is given. The paper concludes with an evaluation of the programme by the trainers, trainees and social scientist involved.     

Meeting the challenge of prescribing and administering medicines safely: structured teaching and assessment for final year medical students

OBJECTIVES: To promote safe prescribing and administration of medicines in the pre-registration house officer (PRHO) year through a programme of structured teaching and assessment for final year medical students. DESIGN: Forty final year medical students from two medical schools were randomly allocated either to participate in a pharmacist facilitated teaching session or to receive no additional teaching. Teaching comprised five practical exercises covering seven skills through which students rotated in small groups. One month later, a random sample of 16 taught and 16 non-taught students participated in a nine-station objective structured clinical examination (OSCE) to assess the impact of the teaching. SETTING: Manchester School of Medicine (MSM), and Kings College School of Medicine and Dentistry (KCSMD). PARTICIPANTS: Final year medical student volunteers. MAIN OUTCOME MEASURES: The need for teaching as indicated by student prior experience; questionnaire rating of student acceptability of teaching and assessment; self-rating of student confidence post-assessment, and student performance assessed by OSCE. RESULTS: The study demonstrated that the taught group achieved higher scores in eight OSCE stations. Four of these were statistically significant (P < or= 0.005). Taught students felt more confident performing the skills on five stations. From 0 to 47.5% students had prior experience of the skills taught. The post-teaching questionnaire evaluated exercises positively on several criteria, including provision of new information and relevance to future work. CONCLUSIONS: Structured teaching provided an effective and acceptable method of teaching the medicines management skills needed in the PRHO year. The structured approach complemented variable precourse clinical experience.     

Audit and summative assessment: two years' pilot experience

An audit project has been submitted by all trainees in the West of Scotland since 1992 as part of a pilot process for summative assessment. The impact of 2 consecutive years of audit was assessed on 117 trainees in May 1994. A response rate of 89% was achieved. For 82 trainees (79%) this was their first practical experience of audit and as a result of it 85% felt more confident in introducing change to their next practice. Protected time was still a problem for the majority (53%) and one-third wanted more help from their trainer. Thirty trainees (29%) had attended four or fewer formal practice meetings in their 10 months of training, with 10% never having attended one. The previous trainee's audit project was rarely or never discussed with 70 trainees (87%) and only 12 trainees were evaluating changes recommended. An audit project as part of summative assessment may be encouraging trainees to think about change after they leave their training practices. Few, however, are evaluating change and protected time and support are still required. The marking of the summative assessment audit project may need to address this.     

The impact of evidence-based education on a perinatal capacity-building initiative in Macedonia

CONTEXT: The perinatal mortality rate (PMR) in Macedonia is among the highest in Europe. The World Bank supported a consultant (HEJ) to collaborate with a Macedonian team to develop a national perinatal strategy with the goal of reducing the PMR. Education was given priority in the form of a hospital-based initiative to develop the capacity of health professionals to introduce evidence-based perinatal practice into 16 participating hospitals. A "train the teachers" approach was used, with trainees introduced to modern education and clinical practice in Sydney and subsequently supported to train their colleagues in Skopje. OBJECTIVES: To describe the development, implementation and evaluation of the educational intervention. METHODS: A curriculum, based on specific Macedonian needs, was developed in order to integrate teaching in the knowledge, skills and attitudinal domains of learning, using small group, interactive techniques. Twenty-five Macedonian doctors and nurses participated in 4-month (phase 1a) and 6-month (phase 1b) teaching programmes at a tertiary perinatal unit in Sydney. Australian staff conducted 4 2-week modules for 36 trainees in Macedonia (phase 2). The phase 1 trainees conducted 8 modules for 57 colleagues in Skopje (phase 3). The intervention was evaluated by trainee questionnaires, assessments of competence, changes in hospital practice and pre- (1997-99) and post-intervention (2000-01) comparisons of PMR. RESULTS: A total of 115 doctors and nurses graduated from the programme. Positive responses to the education programme exceeded 80%. Evidence-based practice in 16 participating hospitals (covering 91% of all Macedonian births) was verified in 6 key areas of neonatology. The PMR fell significantly from 27.4 to 21.5 per 1000 births (RR 0.79, 95% CI 0.73, 0.85). The early neonatal death rate in babies weighing over 1000 g fell by 36%. CONCLUSIONS: The intervention has increased the capacity of Macedonians to practise best-evidence perinatal medicine and improve outcomes. Sustainability is predicted by the "train the teachers" approach, with concurrent strengthening of the infrastructure and organisational framework.     

Educational impact of in-training assessment (ITA) in postgraduate medical education: a qualitative study of an ITA programme in actual practice

Objectives To investigate the experiences and opinions of programme directors, clinical supervisors and trainees on an in‐training assessment (ITA) programme on a broad spectrum of competence for first year training in anaesthesiology. How does the programme work in practice and what are the benefits and barriers? What are the users' experiences and thoughts about its effect on training, teaching and learning? What are their attitudes towards this concept of assessment? Methods Semistructured interviews were conducted with programme directors, supervisors and trainees from 3 departments. Interviews were audiotaped and transcribed. The content of the interviews was analysed in a consensus process among the authors. Results The programme was of benefit in making goals and objectives clear, in structuring training, teaching and learning, and in monitoring progress and managing problem trainees. There was a generally positive attitude towards assessment. Trainees especially appreciated the coupling of theory with practice and, in general, the programme inspired an academic dialogue. Issues of uncertainty regarding standards of performance and conflict with service declined over time and experience with the programme, and departments tended to resolve practical problems through structured planning. Discussion Three interrelated factors appeared to influence the perceived value of assessment in postgraduate education: (1) the link between patient safety and individual practice when assessment is used as a licence to practise without supervision rather than as an end‐of‐training examination; (2) its benefits to educators and learners as an educational process rather than as merely a method of documenting competence, and (3) the attitude and rigour of assessment practice.     

Evaluation of a family-oriented continuing medical education course for general practitioners

OBJECTIVE: To explore the long-term effects of a 2-year Family Systems Medicine course. Fifteen experienced GPs participated in the training programme. SETTING: Continuing Education Centre, University of Tampere, Department of Public Health Science and General Practice, University of Oulu, Finland. METHODS: The participants assessed the development of their professional skills on the Doherty-Baird scale and filled in 2 questionnaires. The material obtained from the application form and 2 questionnaires was analysed using the grounded theory method. RESULTS: The reasons for taking part in the course seemed to be the constant increase in the workload, problems caused by the demands for change and adaptation, stress and exhaustion. Furthermore, 10 health centres out of 15 had adopted the population-based practice, which requires different working methods compared to the old routines. Some trainees reported that their family-centred working methods improved during the course. A year after the end of the training, all of the GPs who had participated were using such methods in their daily practice. Half of the participants felt that they had also improved the functioning of their working group by making it more family-oriented. The significance of multi-professional collaboration was one of the most important insights during the course. CONCLUSION: The 2-year family-oriented training programme provided GPs with systemic thinking and with new skills, including the ability to work with families. The programme raised awareness of the need for multi-professional collaboration in the primary care settings.     

Training the trainers: do teaching courses develop teaching skills?

OBJECTIVE: This paper reports on consultants' self-assessed changes in their teaching and training practices over an 8-10-month period. It compares the changes between a group undergoing a 3-day teaching course (participants) and a sample group taken from the course waiting list (controls). METHOD: A questionnaire listing 18 teaching skills was given to the participants immediately prior to the course and 8-10 months later, and to the controls at the same time intervals. Respondents were asked to rate their ability, frequency of use of each skill, as well as their teaching confidence and effectiveness. Additionally, the second questionnaire asked respondents to identify changes they had made to their teaching. A total of 63% (54) of participants and 51% (23) of controls completed both questionnaires. Changes of 2 + on the rating scales were seen as genuine. The number of such changes was calculated for each individual and on each skill for the 2 groups. Data were analysed using a Mann-Whitney U-test. RESULTS: The majority of course participants reported positive changes in teaching ability on a significantly greater number of skills than did the control group. As a group, changes in ability in 16 of the teaching skills were significantly greater for the participants than for the controls. Increased ability resulted in participants' increased frequency of use of only 4 of the teaching skills. The majority in the participant group reported changes to their teaching. Only a minority in the control group reported such changes. These changes were consistent with course topics and the teaching skills needed to meet General Medical Council recommendations for the education of new doctors. CONCLUSIONS: The teaching course is an effective vehicle for increasing consultants' teaching skills.     

Checklists for assessment and certification of clinical procedural skills omit essential competencies: a systematic review

Objective  To develop generic criteria for the global assessment of clinical procedural competence and to quantify the extent to which existing checklists allow for holistic assessment of procedural competencies.
Methods  We carried out a systematic review and qualitative analysis of published clinical procedural skills assessment checklists and enumerated the contents of each. Source materials included all English-language papers published from 1990 to June 2005, identified from 18 databases, which described or referred to an assessment document for any clinical procedural skill. A pair of reviewers identified key generic themes and sub-themes through in-depth analysis of a subset of 20 checklists with iterative agreement and independent retesting of a coding framework. The resulting framework was independently applied to all checklists by pairs of reviewers checking for the emergence of new themes and sub-themes. Main outcome measures were identification of generic clinical procedural skills and the frequency of occurrence of each in the identified checklists.
Results  We identified 7 themes ('Procedural competence', represented in 85 [97%] checklists; 'Preparation', 65 [74%]; 'Safety', 45 [51%]; 'Communication and working with the patient', 32 [36%]; 'Infection control', 28 [32%]; 'Post-procedural care', 24 [27%]; 'Team working', 13 [15%]) and 37 sub-themes, which encapsulated all identified checklists. Of the sub-themes, 2 were identified after the initial coding framework had been finalised.
Conclusions  It is possible to develop generic criteria for the global assessment of clinical procedural skills. A third and a half of checklists, respectively, do not enable explicit assessment of the key competencies 'Infection control' and 'Safety'. Their assessment may be inconsistent in assessments which use such checklists.     

Peer-assisted learning: a novel approach to clinical skills learning for medical students

OBJECTIVE: This study aimed to determine whether peer-assisted learning (PAL) can enhance clinical examination skills training. METHODS: Three student trainers studied small-group theory and clinical examination and provided PAL as extra tuition for 86 trainees. Trainees watched an examination video, were videotaped practising the examination and, after constructive feedback, repeated the examination. Responses to PAL were evaluated to attain an overview of trainee and trainer performance using visual analogue and Likert scale analyses. Year-group review was undertaken using questionnaires. RESULTS: Trainees evaluated all aspects of PAL highly, including their post-training confidence in examination skills (mean > 7.7 on a 10-cm scale), indicating that the PAL was effective. Written comments confirmed the students perceived the sessions as well structured and of high quality. Compared with trainees in the first groups, those from later groups gave all parameters similar or higher gradings. Those for interest (P = 0.03) and appropriateness (P = 0.01) were significantly higher, suggesting that trainers may improve their technique with time. Students with previous degrees gave similar or lower gradings than standard entry students, with answers about post-training confidence and recommendation to friends being statistically lower (P < 0.006). Six months later, year-group analysis showed that 90% of trainees rated PAL highly, and 86% wished to become trainers. Of the trainers' year group, 79% perceived that PAL training could improve examination skills. CONCLUSIONS: In the context of clinical skills training, PAL was highly evaluated across many parameters, including confidence after training. Student interest and enthusiasm supports suggestions that PAL could be a useful adjunct to clinical skills training.     

The effect of in-training assessment on clinical confidence in postgraduate education

INTRODUCTION: The literature on how in-training assessment (ITA) works in practice and what educational outcomes can actually be achieved is limited. One of the aims of introducing ITA is to increase trainees' clinical confidence; this relies on the assumption that assessment drives learning through its content, format and programming. The aim of this study was to investigate the effect of introducing a structured ITA programme on junior doctors' clinical confidence. The programme was aimed at first year trainees in anaesthesiology. METHODS: The study involved a nationwide survey of junior doctors' self-confidence in clinical performance before (in 2001) and 2 years after (in 2003) the introduction of an ITA programme. Respondents indicated confidence on a 155-item questionnaire related to performance of clinical skills and tasks reflecting broad aspects of competence. A total of 23 of these items related to the ITA programme. RESULTS: The response rate was 377/531 (71%) in 2001 and 344/521 (66%) in 2003. There were no statistically significant differences in mean levels of confidence before and 2 years after the introduction of the ITA programme - neither in aspects that were related to the programme nor in those that were unrelated to the programme. DISCUSSION: This study demonstrates that the introduction of a structured ITA programme did not have any significant effect on trainees' mean level of confidence on a broad range of aspects of clinical competence. The importance of timeliness and rigorousness in the application of ITA is discussed.     

Evaluation of procedural skills training in an undergraduate curriculum

CONTEXT: A substantial proportion of medical students enter their intern year without any basic skills experience. Lack of experience is a significant source of stress for many junior doctors. OBJECTIVES: To evaluate the effect of a basic procedural skills tutorial for Year 3 medical students on their competence in relevant skills at Year 5. SUBJECTS: The control group consisted of 93 medical students who completed Year 3 in 1996. The intervention group consisted of 92 medical students who completed Year 3 in 1997. The intervention group received a practical skills tutorial in Year 3; the control group did not. Both groups were assessed on their practical skills competence during Year 5. METHODS: A 3-hour practical tutorial on injection and suturing techniques was delivered to the intervention group. The effectiveness of the intervention was assessed by self-reported experience of giving injections, inserting sutures and sustaining needlestick injuries, and by teacher-rated competency in four basic procedural skills. RESULTS: Students who received the Year 3 tutorial were significantly more likely to record a satisfactory assessment for their performance in all four basic skills compared with students who did not receive the tutorial. They were less likely than controls to refuse invitations to give injections, but not invitations to insert a suture, during Years 4 and 5. CONCLUSIONS: A single session of formalised teaching in procedural skills in the early stages of a medical degree can have long-term effectiveness in basic skills competence and may increase students' confidence to practise their skills.     

Assessing undergraduate palliative care education: validity and reliability of two scales examining perceived efficacy and outcome expectancies in palliative care

BACKGROUND: Medical students have traditionally received little education in palliative care. However, in 1999, as part of a revised medical curriculum, Year 4 undergraduates at Liverpool University participated in a 2-week programme of education in palliative care. To assess the effect of the education programme, 2 assessment scales were identified: the Self-efficacy in Palliative Care Scale (SEPC) (assessing efficacy in communication, patient management and multiprofessional teamworking) and the Thanatophobia Scale (assessing attitudes towards palliative care). The aim of this study was to examine the psychometric properties of these scales. METHODS: The scales were examined by 5 palliative care doctors for content validity and appropriate wording. Following this, the SEPC and Thanatophobia Scales were completed by the undergraduates (n = 139) prior to and after completion of the education programme. RESULTS: Both scales were analysed independently on pre- and post-test scores. Cronbach's alphas of 0.84-0.85 and 0.92-0.95 were recorded, respectively, indicating high reliability. Varimax rotated principal components analysis of the SEPC Scale suggested 3 distinct factors, as theoretically expected, with high factor loadings of 0.45-0.89 at pre- and post-test. Principal components analysis of the Thanatophobia Scale suggested only 1 factor underlies the scale, as theoretically expected. All 7 items had high factor loadings of 0.60-0.81 at pre- and post-test. CONCLUSIONS: The results suggest that the SEPC and Thanatophobia Scales are valid and reliable assessment scales that may be of use when evaluating the impact of an education programme.     

Portfolio learning for foundation doctors: early feedback on its use in the clinical workplace

CONTEXT: A learning portfolio was developed to support the development of trainee doctors piloting Foundation Programme prototypes across the Northern Deanery in 2004 and 2005. Trainee doctors and their educational supervisors were surveyed about their experiences of using the portfolio in the clinical workplace. METHODS: The evaluation consisted of semi-structured interviews with trainee doctors and supervisors, followed by postal questionnaire surveys in 2004 and 2005. Quantitative and qualitative data were triangulated to identify core findings. RESULTS: Questionnaires were returned from 182/305 (60%) trainee doctors and 104 out of 179 (58%) educational supervisors. The portfolio was felt to be a 'good idea' by 55% supervisors and 48% trainees. Trainees' perceptions of the educational value of the portfolio remained consistently low over 2 surveys and they described a sense of 'burden', whereby they identified problems in workload and usability and in gaining feedback on performance. However, positive trainee attitudes towards the portfolio were significantly correlated with greater perceived educational benefits (r = 0.855, P < 0.001). DISCUSSION: Learning portfolios are now an integral part of Foundation Programme training but this evaluation suggests that many trainee doctors and educational supervisors are yet to be convinced of their educational value. Gaining multi-source feedback, a substantial component of trainee doctors' portfolios, impacts on the wider clinical team and presents a significant challenge to trainees. Educational supervisors continued to rely on feedback from clinical colleagues, rather than portfolio evidence, to monitor trainee doctors' development. Such factors may serve to disengage trainees with the portfolio process by overshadowing any perceived educational gains.     

Individualised training to improve teaching competence of general practitioner trainers: a randomised controlled trial

Introduction In Flanders an important part of training to become a general practitioner (GP) is undertaken within a general practice. This requires a GP trainer to know how to facilitate learning processes. This paper reports a study focused on the research question: Does short but individualised training of GP trainers contribute to their teaching competence? Method A total of 47 GP trainers were randomly divided into 2 groups of a pretest/post-test control group design. After a multiple-station teaching assessment test (MSTAT), the experimental group received a short but individualised training to improve teaching competence while the control group were given no specific programme regarding teaching competence. One year later, all participants were invited to sit the post-test. Results In all, 61 GP trainers participated in the pretest and 51 in the post-test; 44 GP trainers participated in both tests. Despite the large attrition, the internal validity of the experiment was preserved. Beginners gained a median score of 2.04 (on a scale of 0−5) on the first MSTAT. In the post-test, the experimental group (median = 3.12) scored significantly better on the entire test (Mann–Whitney U = 166.5, P < 0.05) compared with the control group (median = 2.84). However, the GP trainers in the control group also appeared to have made progress. Conclusion General practitioners who start as GP trainers have insufficient teaching competence to guarantee good coaching of students. A personal programme leads to progress in teaching competence; however, it would seem that more time and support are necessary to allow GP trainers to gain full teaching competence. The fact that the control group made progress as well suggests that the test had an important learning effect.     

Improving the psychiatric skills of established general practitioners: evaluation of group teaching

Ten established general practitioners, eight of whom were also trainers, took part in a course designed to improve their psychiatric interviewing skills. Participants were instructed in the problem-based model with audiotape and videotape feedback of real consultations in a group setting. Although those attending were experienced practitioners with a particular interest in the management of psychological problems, evaluation demonstrated a significant improvement in their skills after training. It is suggested that trainers could be trained to provide a similar teaching experience for their own trainees.     

Validation of a checklist to assess ward round performance in internal medicine

BACKGROUND: Ward rounds are an essential responsibility for doctors in hospital settings. Tools for guiding and assessing trainees' performance of ward rounds are needed. A checklist was developed for that purpose for use with trainees in internal medicine. OBJECTIVE: To assess the content and construct validity of the task-specific checklist. METHODS: To determine content validity, a questionnaire was mailed to 295 internists. They were requested to give their opinion on the relevance of each item included on the checklist and to indicate the comprehensiveness of the checklist. To determine construct validity, an observer assessed 4 groups of doctors during performance of a complete ward round (n = 32). The nurse who accompanied the doctor on rounds made a global assessment of the performance. RESULTS: The response rate to the questionnaire was 80.7%. The respondents found that all 10 items on the checklist were relevant to ward round performance and that the item collection was comprehensive. Checklist mean-item scores differed between levels of expertise: junior house officers 1.4 (1.0-1.9); senior house officers 2.0 (1.5-2.9); specialist trainees 2.5 (1.8-2.8), and specialists 2.7 (2.3-3.5); median (range) (P < 0.001). A significant correlation was found between global observer scores and nurse scores (r = 0.56, P < 0.001). CONCLUSION: The checklist, developed for assessing trainees' performance of ward rounds in internal medicine, showed high content validity. Construct validity was supported by the higher scores of experienced doctors compared to those with less experience and the significant correlation between the observer's and nurses' global scores. The developed checklist should be valuable in guiding and assessing trainees on ward round performance.