Clarifying the concepts of confidence and competence to produce appropriate self-evaluation measurement scales

INTRODUCTION: This paper reviews the literature on self-evaluation and discusses the findings of a small-scale qualitative study which explored the terms 'confidence' and 'competence' as useful measures in a self-evaluation scale. Four pre-registration house officers took part in interviews and completed a provisional instrument to assess their perceived competence. FINDINGS: Competence and confidence are useful terms for house officers expressing beliefs about their ability to perform their job but the terms should not be used synonymously. In our study, 'competent' represented what individuals knew about their ability and was based on the individual's previous experience of the task. 'Confident' described a judgement which influenced whether an individual was willing or not to undertake an activity. Confidence was not necessarily based on known levels of competence and therefore performance of tasks which were unfamiliar to the house officer also involved the assessment of risk. The authors give examples of task and skill scales which may be useful in the process of self-evaluation by pre-registration house officers. CONCLUSIONS: The authors suggest that the process of assessing oneself is complicated, and by its very nature can never be objective or free from the beliefs and values individuals hold about themselves. Therefore self-evaluation instruments are best used to help individuals analyse their work practices and to promote reflection on performance. They should not be used to judge the 'accuracy' of the individual's evaluation.     

A study of pre-registration house officers' clinical skills

BACKGROUND: Little is known about the ability of pre-registration house officers (PRHOs) to perform basic clinical skills just prior to entering the medical register. OBJECTIVES: To find out whether PRHOs have deficiencies in basic clinical skills and to determine if the PRHOs themselves or their consultants are aware of them. METHOD: All 40 PRHOs at the Chelsea and Westminster and Whittington Hospitals were invited to undertake a 17 station OSCE of basic clinical skills. Each station was marked by one examiner completing an overall global score after completing an itemised checklist. An adequate station performance was the acquisition of a pass/borderline pass grade. Prior to the OSCE, a questionnaire was given to each PRHO asking them to rate their own abilities (on a 5-point scale) in the skills tested. A similar questionnaire was sent to the educational supervisors of each PRHO asking them to rate their house officer's ability in each of the same skills. RESULTS: Twenty-two PRHOs participated. Each PRHO failed to perform adequately a mean of 2.4 OSCE stations (SD 1.8, range 1-8). There were no significant correlations between OSCE performance and either self- or educational supervisor ratings. The supervisor felt unable to give an opinion on PRHO abilities in 18% of the skills assessed. DISCUSSION: This study suggests that PRHOs may have deficiencies in basic clinical skills at the time they enter the medical register. Neither the PRHOs themselves nor their consultants identified these deficiencies. A large regional study with sufficient power is required to explore the generalizability of these concerns in more detail.     

Primary care consultation skills training: implementing a new training programme for senior house officers in an accident and emergency department

Senior house officers (SHOs) in Accident and Emergency (A&E) departments see many patients who present with primary care problems. Until now, most SHOs have lacked postgraduate training in primary care skills to enable them to meet these patients' needs effectively. This paper describes an innovative training programme that has been developing at King's College Hospital, London. It identifies a new opportunity for general practitioners to contribute to the postgraduate medical education of hospital junior medical staff. The training programme was designed to give A&E SHOs protected time in which to reflect on strengths and weaknesses in relation to primary care consultations and learn from their experiences. Its aim was to improve the assessment and management of patients, and to encourage a problem solving approach within the A&E setting. The programme, established in 1992, was developed through collaboration between the departments of A&E Medicine and General Practice and Primary Care. Evaluation has been a central theme in its development, and has been used to ensure that the training meets the needs of each individual set of SHOs and of the department. It has been used in establishing agreement about the training's value and benefits. The authors discuss some of the methodological difficulties encountered in evaluating this type of educational initiative.     

Pre‐registration house officer rotations incorporating general practice: does the order of rotation matter?

CONTEXT: In relation to pre-registration house officer (PRHO) rotations incorporating general practice, previous research has recommended that where possible, no PRHO should undertake general practice as the first placement, because of the difficulties encountered. It was recognized that logistically, this could make such schemes almost unworkable. Within the context of a larger qualitative evaluation comparing how 24 PRHOs learned in hospital and general practice settings, the issue of rotation order was explored. METHODS: In-depth semistructured interviews were conducted with the 12 PRHOs who were involved in general practice rotations. They were interviewed at the beginning and end of the PRHO year, and following their return to hospital work after the general practice placement. RESULTS: Each rotation order had both advantages and disadvantages, with no particular rotation order being obviously better or worse for the PRHOs involved. CONCLUSIONS: This small qualitative evaluation has highlighted a number of advantages and disadvantages specific to each rotation order, and makes some practical recommendations to help alleviate the problems encountered. It is important that future evaluations of similar schemes consider this issue, as there are conflicting reports about the significance of the rotation order.     

The clinical and educational experiences of pre-registration house officers in general practice

AIMS: To describe aspects of the clinical experience and educational supervision gained by pre-registration house officers (PRHOs) in general practice, and to relate these to the current General Medical Council (GMC) aims for general clinical training in general practice. DESIGN: Qualitative evaluation, part of which involved semistructured interviews with 12 PRHOs who were experiencing a general practice rotation. Interviews were conducted at the beginning and the end of the pre-registration year, and following return to hospital work after completion of the general practice placement. SETTINGS: Three teaching hospitals, two district general hospitals and six general practices in south-east England. PARTICIPANTS: 12 PRHOs who were involved in rotations incorporating a general practice placement. RESULTS: To varying degrees, the GMC aims for training in general practice were met for all the participants. All PRHOs recognized the value of the clinical experience and educational supervision they received in general practice. They particularly valued aspects such as having an individual training programme based on their own needs, and the interlinking of theory and practice, which aided learning. Most felt that having responsibility for their own patients acted as an important incentive for learning, and in general, PRHOs appreciated having the time to learn which general practice allowed. CONCLUSIONS: For the majority of PRHOs, the time spent in general practice was seen as a positive clinical and educational experience. In a variety of ways, the general practice placement encouraged PRHOs to develop the self-directed learning skills seen as essential to the lifelong learning advocated by the GMC. A number of recommendations are made to help improve the integration of the hospital and general practice components of these rotations.     

Meeting the needs of future physicians: a core curriculum initiative for postgraduate medical education at a Canadian university

In addition to possessing medical expertise, contemporary physicians are expected to be skilled communicators, critical consumers and users of medical research, teachers, collaborators, health care advocates, and managers. A core curriculum is a common set of learning experiences designed to help prepare physicians for these complex roles. PURPOSE: This article describes the design and implementation of one core curriculum, summarizes the feedback received from residents, and shares some of the lessons we are learning as we use feedback to develop our programme. METHOD: The core curriculum described was implemented at a Canadian university which offers 56 residency programmes with a total enrollment of approximately 360 students. The curriculum consisted of 30 sessions organized around four themes: biostatistics and epidemiology; communications and teaching skills; healthcare management, and ethical, medicolegal and lifestyle issues. Each session in the Core Curriculum was evaluated by residents with respect to the timing, quality, and value of the learning experience. In addition, residents participated in focus group discussions of their Core Curriculum experiences. RESULTS: Key findings related to the characteristics of effective core curriculum learning experiences and to the barriers to implementing a core curriculum across programmes. Of particular salience were findings related to explicit issues of attendance and the diverse needs of learners and programmes, and to more implicit issues of communication and managing change. The specific content and format of the Core Curriculum and the results of the evaluation process will be of interest to others considering a core curriculum for postgraduate medical programmes.     

Evaluating primary care as a base for medical education: the report of the Cambridge Community-based Clinical Course

BACKGROUND: Medical students receive increasing amounts of their clinical education in a primary care setting. The educational possibilities of such attachments are still being explored. AIM: To report the evaluation of a small, radical innovation to provide students with a long-term community attachment which integrated with hospital-based education. METHOD: Between 1993 and 1998, 13 medical students completed 15-month attachments with a single general practice in England. The course offered them experience in the major clinical specialties throughout this period. Students were exposed to specialist as well as generalist education in the context of patients with whom they could establish a continuing relationship. The innovation was evaluated by its feasibility, by students' examination results, by analysis of clinical experience, through formal student feedback and by cost. RESULTS: The course was practicable in a particular setting with academic leadership. The students all passed their exams. They had wide, appropriate clinical experience even though the attachment was to a single practice. When they returned to the hospital environment, students did not feel themselves at a disadvantage compared with traditional students. The costs of the course are controversial: placement costs were higher than in the hospital, but those for facilities were lower. CONCLUSION: It is possible to run a course like this successfully. It remains the most radical attempt to share clinical education in the UK between primary and secondary/tertiary care. Further research is required into providing long-term clinical attachments in NHS primary care settings.     

The effect of the general practice registrar year on perceived skills in palliative care in the West Midlands

OBJECTIVES: To ascertain the effect of 12 months spent as a GP registrar on perceived skills in palliative care. DESIGN: A previously validated questionnaire for use with medical undergraduates is modified and used to survey perceived skills in five aspects of providing palliative care in five different scenarios at two points during the 12-month period of general practice vocational training where no specific teaching intervention is conducted. SETTING: The West Midlands. PARTICIPANTS: 210 GP registrars. RESULTS: Perceived skill ratings were seen to significantly increase during the 12-month period, but anxiety in caring for the dying did not significantly decrease. Ratings of skills were lowest when caring for a child dying with leukaemia or a young adult dying with AIDS. In addition, other important variables which had a statistically significant influence were gender and age, but interestingly not the number of previous senior house officer (SHO) posts undertaken or whether the respondent had had formal teaching on the subject in the past. CONCLUSION: It might therefore be postulated that training as a GP registrar has an important impact on the development of perceived skills in palliative care.     

The current status of senior house officer postgraduate education in a single region

OBJECTIVE: To assess postgraduate education (PGE) for paediatric senior house officers (SHOs) in a single region. METHODS: A survey of all paediatric SHOs in the region was undertaken in the form of a questionnaire and telephone contact. The standard set by the Committee of Postgraduate Medical Deans for SHO education formed the basis for the questionnaire; there should be (i) a designated educational supervisor for each SHO, (ii) regular work appraisal and counselling and (iii) 4 h protected teaching per week. Supplementary questions were asked regarding methods of teaching and study leave. RESULTS: There was a 92% response rate from SHOs and 90% of SHOs contacted had a named educational supervisor. The mean time for protected teaching per week was 2.14 h, although this varied widely between centres. There were many different teaching methods used and 82% of SHOs had no difficulty in obtaining study leave. There was a particular problem for those who were working shifts or cross-covering. CONCLUSIONS: Our study has shown that paediatric SHOs in Wales are reasonably satisfied with their postgraduate education, although there is considerable variation between different units. Several areas have been highlighted which need to be improved if we are to provide SHOs with adequate preparation for the specialist registrar training grade.     

The development and evaluation of a personal learning log for Senior House Officers

OBJECT: To develop and evaluate the effect of having a personal learning log on Senior House Officers knowledge and confidence. METHODS: A multiple choice paper and a confidence checklist for two hospital specialties were developed to assess knowledge and confidence. These were administered to a control group and to an intervention group who had the learning log. Both groups completed an evaluation at the end of the post. SETTING: The study took place in Accident & Emergency and Obstetric & Gynaecology posts in Greater Glasgow and Lanarkshire. SUBJECTS: 79 Senior House Officers in Accident & Emergency and 78 Senior House Officers in Obstetrics & Gynaecology. RESULTS: The mean scores in the MCQ and the mode in the confidence checklist increased significantly in both specialties during the post, but there was no significant difference between the control and intervention groups. Forty two learning logs were returned at the end of the study and analysis of these revealed that there was great scope for learning but few documented the specific learning achieved. Evaluation of the posts revealed that some improvements had taken place in teaching and assessment frequency, however, there was scope for further improvement. CONCLUSION: While the problems of hospital training are well documented, an attempt to improve the situation using a learning log did not have a statistically significant impact on SHO knowledge or confidence. A six-month hospital post appears to present many opportunities for learning but these are not exploited. It is suggested that three things are needed. Firstly, active participation by and personalized feedback from a senior member of staff, with training where needed. Secondly, protected time for tutorials with a planned system of formative assessment, and thirdly, a more positive approach to learning by both SHOs and consultants. Once this occurs, a learning log may have a more significant impact on training.     

Reliability of the MRCP(UK) Part I Examination, 1984-2001

OBJECTIVES: To assess the reliability of the MRCP(UK) Part I Examination over the period 1984-2001, and to assess how the reliability is related to the difficulty of the examination (mean mark) and to the spread of the candidates' marks (standard deviation). METHODS: Retrospective analysis of the reliability (KR20) of the MRCP(UK) examination recorded in examination records for the 54 diets between 1984 and 2001. RESULTS: The reliability of the examination showed a mean value of 0.865 (SD 0.018, range 0.83-0.89). There were fluctuations in the reliability over time, and multiple regression showed that reliability was higher when the mean mark was relatively high, and when the standard deviation of the marks was high. CONCLUSIONS: The reliability of the MRCP(UK) Examination was maintained over the period 1984-2001. As theory predicted, the reliability was related to the average mark and to the spread of marks.     

Identifying core skills for the medical curriculum

A survey was undertaken at the University of Sheffield Medical School to identify clinical and other professional skills which are required on graduation and to assess whether and when these skills have been acquired. The purpose of the survey was to identify core practical skills, defined as skills which the majority of clinical firms require and which, in addition, were used by the majority of Pre-registration house officers (PRHOs). The survey was conducted by questionnaires which were circulated to the following four sample groups (number in sample: % response rate): (1) consultants in clinical firms taking undergraduates and post-graduates from Sheffield Medical School (63: 75%); (2) house officers starting their pre-registration year (118: 52%); (3) house officers ending their pre-registration year (120: 43%); and (4) undergraduates starting their final year (110: 65%). The questionnaire contained a stimulus list of 31 clinical and four personal skills with an opportunity for respondents to list other skills they felt to be important. Of the 35 skills listed on the questionnaire, 26 were identified as core practical skills. Fifteen of the core skills had been acquired by the majority of students by the time they graduated, 13 of which were acquired prior to the final undergraduate year. Core skills not acquired as an undergraduate were acquired as a PRHO. The survey acted as a quality control mechanism for teaching at both undergraduate and post graduate levels and provided the basis of a medical school core of practical skills on the basic medical education continuum.     

Early patient contact in primary care: a new challenge

BACKGROUND: The Medical School of Lund University, Sweden, has introduced an early patient contact course, including training in communication and examination skills. The course runs parallel with theoretical subjects during the students' first two-and-a-half years. General practitioner (GP) participation is gradually increasing, and in the last half-year of the course GPs in all health centres in the area are involved. Little is known about the GPs' interest, competence and time for this new task. AIM: To describe the GPs' attitudes towards teaching and the rewards and problems they experience. SUBJECTS: 30 GPs teaching third-year medical students. METHOD: Semistructured interview study. Data analysis by a method described by Malterud. RESULTS: The attitude towards teaching was mostly positive and the teachers were confident about teaching examination procedure. Among rewards of teaching, improved quality of clinical practice was the main theme, but imparting knowledge to others, contact with enthusiastic students, and gains in self-esteem were also mentioned. Problems with teaching were mostly due to external factors such as lack of time and space, but concern about a negative effect on patient care was also recognized. Educational objectives of the course were not completely accepted. GPs were not fully aware about what to expect from the students, with subsequent problems concerning how to assess students' performance and how to give effective feedback. CONCLUSIONS: The teaching of junior medical students is maintained by the GPs' enthusiasm for teaching. However, teacher training is required and the crucial issues of time and space have to be considered.     

Education and training in the senior house officer grade: results from a cohort study of United Kingdom medical graduates

INTRODUCTION: This paper shows the findings from a survey of 439 senior house officers undertaken as part of the British Medical Association cohort study of 1995 medical graduates. The aim of the study was to assess the quality of senior house officer training in the United Kingdom. METHOD: In July 1997 a postal questionnaire was sent to a sample of 545 doctors who graduated from medical school in 1995. Responses were received from 515 (95%). Only those doctors who had worked as a senior house officer in the previous 12 months were included in the analysis (n = 439). RESULTS: Encouraging results are that 69% of the senior house officers surveyed had discussed their progress directly with their consultant, and 24% rated their supervision by their consultant as 'excellent'. Of concern are the findings that 47% of respondents did not receive protected teaching time and 16% were unable to take study leave. DISCUSSION: The study revealed wide variability in the quality of training received by senior house officers in the United Kingdom. Whilst some respondents - notably those in general practice, accident and emergency, paediatrics and psychiatry - had enjoyed a high standard of education and training, it was clear that a minority of posts continue to offer little if any educational value to the post holder. The results point to a need for a more systematic approach to maintaining standards in senior house officer training with greater incentives for under-performing trusts.     

The impact of formal instruction in clinical examination skills on medical student performance -- the example of peripheral nervous system examination

BACKGROUND: Peripheral nervous system examination is an essential part of the full medical clerking of a patient. We have investigated the effectiveness of formal instruction in peripheral nervous system examination compared to the traditional bedside ward teaching that our students usually receive. METHOD: We instructed an unselected group of 22 medical students in peripheral nervous system examination in a clinical skills centre and evaluated them with a 12 item marking schedule before and after instruction. The performance of this group was then compared to the rest of their year (220 students) in an end of year OSCE, which included a neurology station assessing sensory examination of the lower limbs. RESULTS: Students formally instructed in neurology significantly improved their scores after instruction and scored 15% higher marks (90% vs. 75%) than the rest of their year in the end of year neurology OSCE station 2 months later (P < 0.01, Mann Whitney U-test). They did not perform significantly better in the OSCE overall. CONCLUSIONS: Formal instruction in neurological examination resulted in a significant increase in the end of year neurology OSCE station score compared to traditional heterogeneous teaching methods.     

Effective teaching of communication skills for medical practice: selecting an appropriate clinical context

Increasingly, courses in communication skills are being incorporated into medical training. In order for communication skills to be effectively maintained in post-training medical practice, they must be taught within an appropriate clinical context. The present paper describes and provides rationale for seven criteria by which to select clinical issues which are appropriate foci for communication skills courses. The criteria are : (1) the issue must be one which is encountered frequently in clinical practice; (2) the issue must be associated with a high burden of illness; (3) there must be evidence that practitioners need to improve skills for dealing with the issue; (4) there must be an intervention, of which communication skills are an integral component, that is demonstrably effective for dealing with the clinical issue; (5) the intervention must represent a cost-effective means of dealing with the issue; (6) the intervention must be acceptable to doctors and be able to be incorporated into routine medical practice; (7) the intervention must be acceptable to patients. Examples of clinical issues which fit these criteria are given in the paper and include smoking, hazardous alcohol consumption, non-adherence to treatment instructions, overdue cervical screening, inappropriate diet, recovery from medical interventions, and breaking bad news to patients.     

Content validity of a trainer's report: summative assessment in general practice

A structured trainer's report for use in summative assessment in general practice has been designed, and it is important that the content validity of this report is tested before it is widely adopted. The aim of this paper was to assess the content validity of a proposed report from the perspective of doctors who have recently completed vocational training. A postal questionnaire survey was sent out seeking the views of 220 doctors who had completed vocational training in 1994 as to whether or not the proposed contents were items that are needed in general practice, and whether or not the items were suitable for assessment by means of a trainer's report. The response rate was 73.3%. More than 85% of respondents agreed that the 31 items proposed for inclusion in the trainer's report were needed in general practice. While the proportions agreeing that the items were suitable for assessment by a trainer's report were considerably lower, for 26 items significantly more agreed that they were suitable than disagreed. Analysis of the freetext comments made by respondents illustrated their concerns. The results provide support for the content validity of this proposed trainer's report; however it is important that the concerns expressed are addressed.     

Experiences from psychiatric rotation for pre-registration house officers: contributions to subjective learning

SETTING: The clinical experience and the subjective learning of pre-registration house officers who completed 4 months' psychiatric rotation are described. METHOD: The study was cross-sectional and based on self-report. The participants had responded to an anonymous questionnaire at the completion of their 4 months' psychiatric rotation in the period 1999-2001. Of those eligible (n = 137), 85 (61%) completed the questionnaire. The relationships between individual background variables, different factors of the learning environment, and good learning outcome were explored. CONCLUSIONS: The subjective learning benefit and positive learning environment were most related to the amount of practical experience, the degree of tailoring of the tasks to the house officers' competence, and to the formal intramural teaching programme. Supervision, gender, previous experience, and prior interest in psychiatry did not have significant impact on subjective learning. The findings are discussed in terms of the learning model that was developed.     

Effective supervision in clinical practice settings: a literature review

CONTEXT: Clinical supervision has a vital role in postgraduate and, to some extent, undergraduate medical education. However it is probably the least investigated, discussed and developed aspect of clinical education. This large-scale, interdisciplinary review of literature addressing supervision is the first from a medical education perspective. PURPOSE: To review the literature on effective supervision in practice settings in order to identify what is known about effective supervision. CONTENT: The empirical basis of the literature is discussed and the literature reviewed to identify understandings and definitions of supervision and its purpose; theoretical models of supervision; availability, structure and content of supervision; effective supervision; skills and qualities of effective supervisors; and supervisor training and its effectiveness. CONCLUSIONS: The evidence only partially answers our original questions and suggests others. The supervision relationship is probably the single most important factor for the effectiveness of supervision, more important than the supervisory methods used. Feedback is essential and must be clear. It is important that the trainee has some control over and input into the supervisory process. Finding sufficient time for supervision can be a problem. Trainee behaviours and attitudes towards supervision require more investigation; some behaviours are detrimental both to patient care and learning. Current supervisory practice in medicine has very little empirical or theoretical basis. This review demonstrates the need for more structured and methodologically sound programmes of research into supervision in practice settings so that detailed models of effective supervision can be developed and thereby inform practice.