Clinical skills in junior medical officers: a comparison of self-reported confidence and observed competence

BACKGROUND: The intern year is a key time for the acquisition of clinical skills, both procedural and cognitive. We have previously described self-reported confidence and experience for a number of clinical skills, finding high levels of confidence among Australian junior doctors. This has never been correlated with an objective measure of competence. AIMS AND HYPOTHESIS: We aimed to determine the relationship between self-reported confidence and observed competence for a number of routine, procedural clinical skills. METHODS: A group of 30 junior medical officers in their first postgraduate year (PGY1) was studied. All subjects completed a questionnaire concerning their confidence and experience in the performance of clinical skills. A competency-based assessment instrument concerning 7 common, practical, clinical skills was developed, piloted and refined. All 30 PGY1s then completed an assessment using this instrument. Comparisons were then made between the PGY1s' self-reported levels of confidence and tutors' assessments of their competence. RESULTS: A broad range of competence levels was revealed by the clinical skills assessments. There was no correlation between the PGY1s' self-ratings of confidence and their measured competencies. CONCLUSIONS: Junior medical officers in PGY1 demonstrate a broad range of competence levels for several common, practical, clinical skills, with some performing at an inadequate level. There is no relationship between their self-reported level of confidence and their formally assessed performance. This observation raises important caveats about the use of self-assessment in this group.     

The content of the medical curriculum at McMaster University: graduates'' evaluation of their preparation for postgraduate training

This paper describes McMaster University medical graduates' perceptions of how well their medical curriculum prepared them for postgraduate training. The graduates view their overall preparation for postgraduate work as sound. These perceptions were compared with independent assessments by internship supervisors for one graduated class. The graduates suggest their preparation for postgraduate work differs somewhat from fellow interns. Graduates reported feeling very well prepared compared to fellow postgraduate trainees in independent learning, self-evaluation and problem solving skills. They also judge their preparation in data gathering skills, behavioural science knowledge, ability to deal with social and emotional problems of patients, medical record keeping skills, preventive, follow-up and in-patient care as very good compared to peers. They identified two content areas, pharmacology and the basic medical sciences, as requiring more attention in the curriculum. These findings are discussed and related to the approach to medical education at McMaster University.     

Growth of Self-Perceived Clinical Competence in Postgraduate Training for General Practice and its Relation to Potentially Influencing Factors

Objective To examine the increase in self-perceived clinical competence during a three-year postgraduate training in general practice and to explore the relation between the growth of self-perceived competence and several background variables. Design Cohort, 1995–1998. Setting Three-year Postgraduate Training for General practice in the Netherlands Participants All Dutch trainees who followed postgraduate training from September 1995 to September 1998 (N=191). Intervention We asked the trainees at the start and at the end of their postgraduate training to complete a questionnaire, which assessed their self-perceived knowledge, clinical skills and consultations skills. We collected information about potentially influencing background variables. Amongst these were variables such as: age, gender, prior medical experience, the effort someone has spent upon her/his education, insight in weak and strong areas of clinical competence and knowledge and skills levels. Main outcome measure Self-perceived competence. Results A total of 127 trainees completed both questionnaires (190 at the first administration and 128 at the second one). We found statistically significant growth of self-perceived clinical competence. Self-perceived consultation skills increased more than self-perceived knowledge and clinical skills. The afore mentioned background variables did not relate in any way with the growth of self-perceived clinical competence. Conclusion This study shows that a 3-year postgraduate training in general practice enhances self-perceived clinical competence. However, we still do not know how to explain this improvement. Further study into the theoretical concept of self-assessment in medical education and into the factors contributing to the feeling of being competent, is required.     

The effect of multimedia replacing text in resident clinical decision-making assessment

Multimedia in assessing clinical decision-making skills (CDMS) has been poorly studied, particularly in comparison to traditional text-based assessments. The literature suggests multimedia is more difficult for trainees. We hypothesize that pediatric residents score lower in diagnostic skill when clinical vignettes use multimedia rather than text for patient findings. A standardized method was developed to write text-based questions from 60 high-resolution, quality multimedia; a series of expert panels selected 40 questions with both a multimedia and text-based counterpart, and two online tests were developed. Each test featured 40 identical questions with reciprocal and alternating modality (multimedia vs. text). Pediatric residents and rising 4th year medical students (MS-IV) at a single residency were randomized to complete either test stratified by postgraduate training year (PGY). A mixed between-within subjects ANOVA analyzed differences in score due to modality and PGY. Secondary analyses ascertained modality effect in dermatology and respiratory questions using Mann–Whitney U tests, and correlations on test performance to In-service Training Exam (ITE) scores using Spearman rank. Eighty-eight residents and rising interns completed the study. Overall multimedia scores were lower than text-based scores (p = 0.047, η _p ²  = 0.04), with highest disparity in rising interns (MS-IV); however, PGY had a greater effect on scores (p = 0.001, η _p ²  = 0.16). Respiratory questions were not significantly lower with multimedia (n = 9, median 0.71 vs. 0.86, p = 0.09) nor dermatology questions (n = 13, p = 0.41). ITEs correlated significantly with text-based scores (ρ = 0.23–0.25, p = 0.04–0.06) but not with multimedia scores. In physician trainees with less clinical experience, multimedia-based case vignettes are associated with significantly lower scores. These results help shed light on the role of multimedia versus text-based information in CDMS, particularly in less experienced clinicians.     

Acquisition of communication skills in postgraduate training for general practice

PURPOSE: The evidence suggests that a longitudinal training of communication skills embedded in a rich clinical context is most effective. In this study we evaluated the acquisition of communication skills under such conditions. METHODS: In a longitudinal design the communication skills of a randomly selected sample of 25 trainees of a three-year postgraduate training programme for general practice were assessed at the start and at the end of training. Eight videotaped real life consultations were rated per measurement and per trainee, using the MAAS-Global scoring list. The results were compared with each other and with those of a reference group of 94 experienced GPs. RESULTS: The mean score of the MAAS-Global was slightly increased at the end of training (2.4) compared with the start (2.2). No significant difference was found between the final results of the trainees and the reference group. According to the criteria of the rating scale the performance of both trainees and GPs was unsatisfactory. CONCLUSION: The results of this study indicate that communication skills do not improve in a three-year postgraduate training comprising both a rich clinical context and a longitudinal training of communication skills, and that an unsatisfactory level still exists at the end of training. Moreover, GPs do not acquire communication skills during independent practice as they perform comparably to the trainees. Further research into the measurement of communication skills, the teaching procedures, the role of the GP-trainer as a model and the influence of rotations through hospitals and the like, is required.     

The views of senior students and young doctors of their training in a skills laboratory

The study was carried out in the Faculty of Medicine and Health Sciences (FMHS), United Arab Emirates (UAE) University, UAE, where a skills laboratory was set up in 1988 to train medical students in clinical skills before they use such skills on patients. The students learn clinical skills using simulated patients, models and mannequins. The training starts in the first year of the 6-year undergraduate curriculum and continues until the end of the fourth year, after which students rotate through the clinical specialties. This study sought to identify: (1) the views of senior clerks (sixth year) and interns (first postgraduate year) regarding the clinical skills training in the skills laboratory (SKL) during the early years of the medical curriculum; (2) the differences in views between senior clerks and interns; (3) the differences in views between women and men students. Students' views about communication, interviewing, physical examination, therapeutic, diagnostic and laboratory skills were investigated. The results indicated that prior training in clinical skills was helpful to the senior clerks and the interns. Statistically significant differences in opinion were noted between the senior clerks and the interns, the interns being more positive about the usefulness of the SKL training compared with learning clinical skills directly on patients. There was no difference between men and women students except that women students were not comfortable with allowing their peers to examine any part of their bodies. The findings of the study have direct implications for the SKL programme.     

Clinical teachers as role models: perceptions of interns and residents in a Lebanese medical school

OBJECTIVE: To identify the characteristics and learning impact of role models as perceived by interns and residents in an Arabic Middle Eastern country, Lebanon. METHODS: A structured and self-administered questionnaire was sent to the cohort of interns (n = 34) and residents (n = 66) training in a Lebanese university hospital. The questionnaire contained pre-specified items related to professional and personal characteristics of positive and negative role models, as well as to the impact of these models on professional learning and career choices. Responders were asked to recognise and to rank-order the items associated to their identified models. RESULTS: A total of 88 responders (97%) had positive role models and 87 responders (96%) had negative role models in their current training programme. Characteristics identified most frequently and ranked most highly by the trainees were related to clinical skills in positive role models and to inadequate humanistic and collaborative attitudes in negative models. Role modelling had a positive impact on the achievement of clinical skills for 55% of the responders, and on the acquisition of humanistic and collaborative attitudes for 30% of them. Thirty-eight per cent of the trainees were influenced by their role models in the choice of their specialities. Responses were generally comparable between levels of training and between medical and surgical specialities. CONCLUSION: Clinical teachers, practising in a non-occidental cultural and medical learning environment, are considered very frequently as role models by their interns and residents and have a positive impact on their professional learning and career choices. Strategies to reinforce role modelling are needed in Lebanon, in order to pass a broad spectrum of professional values to our students.     

Educational assessment of general practice experience for psychiatric trainees

Eighteen psychiatric trainees spent 6 months each as general practice trainees. The educational impact of the experience was assessed by a self-assessment questionnaire, a semi-structured interview and a videotaped interview with a psychiatric patient. Each assessment was conducted at baseline and after 12 months. A control group of 14 trainees was recruited from the same rotation. On the self-assessment questionnaire, the study registrars rated their abilities to solve general medical problems significantly improved compared to controls. They had also acquired greater understanding of the limitations of their knowledge and their legal responsibilities towards their patients. The semi-structured interview failed to distinguish between the two groups. Videotapes for rating at baseline and follow-up were available for only 17 of the trainees. Assessment of the tapes used the Maguire Scale and the Interview Behaviour Scale. Neither scale demonstrated any intervention effect. The interviews were all characterized by a preponderance of 'closed psychological' and 'checking-out' questions. It appears that psychiatric trainees' interviewing styles had not been influenced by the experience. This study suggests that psychiatric trainees gain greater confidence in their role as a doctor and greater understanding of the scope and nature of general practice by such an attachment. It is unclear whether or not supplementary interviewing skills had been acquired which were not utilized in the taped interview, which conforms very much to traditional psychiatric examination behaviour. Trainees were reassured that they had increased their knowledge without losing any of their specific professional skills.     

Successful teaching in evidence-based medicine

OBJECTIVES: Several published articles have described the importance of exposing medical trainees to the 'new paradigm' of evidence-based medicine (EBM). Recognizing this, we sought to develop and objectively evaluate a mini-course in EBM for third-year medical students. DESIGN: We developed a mini-course consisting of four sessions in which students learn to derive sequentially focused questions, search MEDLINE, review articles critically and apply information from the literature to specific clinical questions. To evaluate the teaching intervention, we performed a controlled educational study. Students at the intervention site (n=34) attended the EBM mini-course, while students at the control site (n=26) received more 'traditional' didactic teaching on various clinical topics. Intervention and control students were surveyed immediately before and after the mini-course to assess changes in reading and literature searching skills, as well as a tendency to use the literature to answer clinical questions. SETTING: Boston University School of Medicine. SUBJECTS: Third-year medical students. RESULTS: The intervention was associated with significant changes in students' self-assessed skills and attitudes. MEDLINE and critical appraisal skills increased significantly in the intervention group relative to the control group (significance of between group differences: P=0.002 for MEDLINE and P=0.0002 for critical appraisal), as did students' tendency to use MEDLINE and original research articles to solve clinical problems (significance of between group differences: P=0.002 and P=0.0008, respectively). CONCLUSIONS: We conclude that this brief teaching intervention in EBM has had a positive impact on student skills and attitudes at our medical school. We believe that the key elements of this intervention are (1) active student involvement, (2), clinical relevance of exercises and (3) integrated teaching targeting each of the component skills of EBM.     

Learning together: clinical skills teaching for medical and nursing students

OBJECTIVE: To evaluate the feasibility and effectiveness of shared learning of clinical skills for medical and nursing students at the University of Manchester. DESIGN: Medical and nursing students learned clinical skills in either uniprofessional or multiprofessional groups. These groups rotated through skills stations taught by multiprofessional facilitators. The groups stayed together for a series of 3 sessions held at weekly intervals (an induction meeting followed by 2 3-hour teaching sessions). Both quantitative and qualitative methods were used to evaluate the project. CONTEXT: A total of 113 Year 3 students registered on the Medical (n = 41), Bachelor of Nursing (Honours) (n = 43) or the Diploma for Professional Studies in Nursing (n = 29) courses participated in the project which was set in the clinical skills unit of a teaching hospital. RESULTS: Pre- and post self-evaluation of confidence levels for the taught skills revealed a statistically significant increase for all skills. The primary reason students gave for participation in the project was to learn or consolidate skills. An additional inducement for participation was the opportunity to share knowledge and observations between professional groups. Tutors also evaluated the experience favourably, particularly with regard to small group discussions. They indicated that the programme provided an opportunity to standardise clinical skills teaching. CONCLUSION: Collaborative learning opportunities for nursing and medical students are feasible and add value to the learning experience. Data indicate positive outcomes of learning in multiprofessional groups, comprising increased confidence levels, increased understanding of others' professional roles and personal development.     

Trainees assessment of vocational training in psychiatry: a pilot study at the Maudsley Hospital

The adequacy of postgraduate trainees is assessed rather more frequently than that of the teaching process. A survey was therefore undertaken of the views of sixty-two junior psychiatrists regarding the clinical units they had worked on during their 3-year rotational training programme at the Maudsley Hospital. The quality of the training experience was felt to be related to work load (not too light), to the number of trainees attached to each consultant, and to the attention given by the consultant to instruction on practical management. The factors felt to need most attention were feedback from consultants on the trainee's performance, and the standards of academic instruction and of interdisciplinary teamwork.     

Factors influencing the communication skills of first-year clinical medical students

The aim of this study was to examine the factors influencing medical students' communication skills. The sample comprised all first-year clinical students. Thirty-two received teaching in communication skills during the year; the remaining 56 did not. Students' career preferences, attitudes towards communication skills and confidence in their ability to communicate with patients were assessed by questionnaire at the beginning and end of the year. At the end of the year each student was videotaped interviewing a simulated patient. Students' communication skills were assessed on the basis of this interview by raters using a standardized rating scale, and by patient questionnaires. While there was some evidence that brief communication skills training improved skills, sex of student was a more significant predictor of level of skill. Students who perceived communication skills as less relevant to medicine and those who were more confident about their own communication skills were more likely to prefer a career in hospital medicine. Students' judgements of their ability to communicate effectively were poor. In the main there was no relationship between confidence and level of skill: where they were related, the association was negative. The benefits from communication skills training might be enhanced by involving hospital doctors in the teaching, and providing students with detailed video feedback on their skills at the outset.     

Comparison of training techniques using a patient-centred approach to smoking cessation

This study compared the use of a 2.1/2-hour multimedia workshop with distribution of an algorithm on the ability of fourth-year medical students to present a stop-smoking plan to a simulated patient. Results showed that students who participated in the workshop performed statistically significantly better on the skill areas of providing information, eliciting and responding to feeling and on content areas of past experience with quitting, resources available for change and negotiating a plan. There were no significant differences in the skill area of eliciting information and the content areas of motivation to stop smoking, factors that inhibit change and problems affecting the plan. Neither of the groups performed very well. The highest number of available points obtained by both groups was in eliciting information (53% in the algorithm group and 64% in the formal training group); however, most of the values were in the range of 10%–25% of possible points. Suggested reasons for the low values may be due to the specific items rated, the teaching methods or the time needed to assimilate new skills.     

Comparison between medical students' experience,confidence and competence

OBJECTIVES: This study was undertaken to determine whether or not breadth of clinical experience and student levels of confidence were indicators of competency on standardized simulator performance-based assessments. METHODS: All students (n=144) attending an educational session were asked to complete a 25-point questionnaire regarding specific clinical experiences and levels of confidence in their ability to manage patient problems. For enumeration of clinical experiences, students were asked to estimate the number of times a situation had been encountered or a skill had been performed. For level of confidence, each response was based on a 5-point Likert scale where 1=novice and 5=expert. Students then participated in a standardized simulated performance test. Median and range were calculated and data analysed using Spearman rank correlations. A P-value <0.05 was considered significant. Level of confidence data were compared to performance during clinical rotation and to marks in the anaesthesia final examination. RESULTS: A total of 144 students attended the session, completed the questionnaire and participated in the standardized test. There were wide ranges of experience and confidence in the 25 listed items. Analysis of data showed good correlation between clinical experience and level of confidence. There was no correlation between clinical experience, level of confidence and performance in a standardized simulation test. Neither was there any correlation between level of confidence and clinical grades or written examination marks. CONCLUSIONS: Clinical experience and level of confidence have no predictive value in performance assessments when using standardized anaesthesia simulation scenarios.     

Patients teach students: partners in arthritis education

CONTEXT: A large metropolitan teaching hospital within The Northern Clinical School, University of Sydney. OBJECTIVE: To assess whether students taught by trained patients (Patient Partners) acquire the same levels of competence in musculoskeletal examination skills for arthritis as students taught by Consultant Rheumatologists. SUBJECTS: Year four medical students in a six-year Undergraduate Medical Programme. METHOD: Students randomized to eight tutorial groups were taught musculoskeletal examination skills in a 75-90 minute tutorial. Four groups were taught by Consultants with an untrained patient present and four groups were taught by Patient Partners. RESULTS: Students' mean self-ratings of skill before and after their tutorial were summed. For both groups, self-ratings before the tutorial were similar. After the tutorial both groups showed substantial gains in levels of skill. Patient Partners' ratings of students' taught by either Consultants or Partners were comparable. CONCLUSIONS: Patient Partners are at least equal to Consultant Rheumatologists in the teaching of musculoskeletal examination techniques for arthritis.     

An audit of clinical teaching in paediatric surgery to interns and surgical registrars

OBJECTIVES: The study aimed to assess the knowledge increment in paediatric surgery of interns (pre-registration) after a 1-month period of training and of registrars after a 6-month rotation. A comparison of the knowledge base of interns from different universities was included. DESIGN: A standard questionnaire was completed by all interns and registrars on the first day of their appointment and again at the end of rotation. Knowledge increment was assessed for each student and each question. SETTING: King Edward VIII Hospital, Durban, South Africa. SUBJECTS: Interns (equivalent to pre-registration house officers) and registrars (registered practitioners) undergoing general surgical training. RESULTS: Both registrars and interns improved their test scores after their training period. However, satisfactory exit scores were achieved by interns in only 72% of questions. CONCLUSIONS: This study forms the basis for assessing future educational strategies and has identified areas of teaching weakness which can be remedied. The reduction in exposure of interns to clinical paediatric surgery must be balanced by more efficient use of teaching time.     

Community-based projects in rural internships: an alternative approach

Supervision of medical interns posted to various primary health centres and rural health training centres by specialists in preventive and social medicine and other clinical disciplines is becoming less and less effective for a number of unavoidable reasons. Because of lack of proper and timely guidance, interns feel that during the 6-month rural internship they do not get enough experience of rural life. In order to provide them with learning experiences in community medicine and orient them in the social dynamics of the community, a new approach involving interns in small community-based projects, probably for the first time, was tried on a pilot basis at the Rural Health Training Centre (RHTC), Sirur, a field practice area of B.J. Medical College, Pune, Maharashtra, India. Interns working at RHTC Sirur completed these community-based projects successfully. Identification of problems, study design analysis and drawing conclusions, based on observation, were all undertaken by the interns under the guidance of the staff of the Department of Preventive and Social Medicine, B.J. Medical College. The opinion poll at the end of the rural internship revealed that 76% of interns considered this experience valuable for improving their knowledge and skills, and 56% though that interaction during these projects was beneficial to the community as well. This experience with community-based projects for interns during their rural posting provides them with an opportunity for interaction with the community.     

Retention of interviewing skills learned by first-year medical students: a longitudinal study

A pilot interviewing course was offered as an elective to first-year medical students in the spring of 1985. A quasi-experimental pre-test/post-test control group demonstrated that students who took the course exhibited a significant increase in interviewing skills. Subjects, 15 in the experimental group and 11 in the control group, were assessed in each of the following 3 years. Data consisted of five 10-minute videotaped interviews with real or stimulated patients for each subject and subject's responses on a degree of confidence, familiarity and anxiety scale. Interviews were rated on 43 behaviours by two independent coders with a 90% simple agreement. The experimental group did not maintain their scores on interviewing skills and both groups showed a significant decline in nine skills comprising empathy. The only significant difference between the groups in fourth year was on the degree of confidence experienced. While interviewing skills can be learned they decline in the clinical years as students learn medical problem-solving. If medical students are to graduate with their original empathy intact, a follow-up course in fourth year is indicated.     

Nutrition care planning: comparison of the skills of dietitians, interns, and students

A written case study--or patient management problem--was used to examine how the nutrition care planning process is influenced by increasing clinical experience. The sample consisted of 45 students in diet therapy courses, 46 dietetic interns, and 44 experienced clinical or generalist dietitians. The patient management problem tested data gathering, with sections containing information needed to assess nutritional status, and nutrition care planning, in which subjects identified and evaluated problems and selected short- and long-term goals. Performance was scored by comparing subjects' answers with those of five dietetic experts. Dietitians scored highest and students scored lowest on efficiency of data gathering. Dietitians and interns scored higher than students on proficiency of data gathering and nutrition care planning. Overall, dietitians and interns performed better than students. Only in tasks requiring the highest levels of information processing and/or confidence in clinical judgment were differences between dietitians and interns consistently found. These findings demonstrate that basic nutrition care planning skills are acquired during dietetic internships, but advanced skills are added with clinical experience.     

A survey of threats and violent acts by patients against training physicians

OBJECTIVES: The primary purpose was to determine the prevalence of various types of threats or assaults by patients against training physicians and to determine the psychological impact of the most distressing incidents. Differences between specialty of training and gender were examined. DESIGN: An anonymous mailed questionnaire. SETTING: The Medical School of the University of Auckland, New Zealand. SUBJECTS: All 160 postgraduate trainees in psychiatry, general medicine, surgery, and obstetrics and gynaecology. A response rate of 84% (n=135) was obtained. RESULTS: The majority of trainees had been verbally threatened (n=91, 67%) or physically intimidated (n=73, 54%) at some time during specialty training, while another 41% (n=55) had witnessed Health Board property being damaged in their presence and 39% (n=53) had been physically assaulted. Psychiatry trainees were significantly more likely to experience the various types of threat or violence. Females (n=20, 38%) were significantly more likely than men (n=8, 10%) to report having been sexually harassed (chi2=14, d.f.=1, P < 0.001). The overall mean on the Impact of Event Scale for those who described the most distressing incident was 8. On only one occasion was a training director directly informed about a trainee's most distressing incident and most trainees (n=95, 70%) had not had training on protecting against assault or on managing violence. CONCLUSION: These findings underscore a priority for developing programmes which effectively reduce threats and violence against trainees and which lessen the psychological sequelae of these incidents.