Reliability and validity of the objective structured clinical examination in paediatrics

The assessment of clinical competence has traditionally been carried out through standard evaluations such as multiple choice question and bedside oral examinations. The attributes which constitute clinical competence are multidimensional, and we have modified the objective structured clinical examination (OSCE) to measure these various competencies. We have evaluated the validity and reliability of the OSCE in a paediatric clinical clerkship. We divided the examination into the four components of competence (clinical skills, problem-solving, knowledge, and patient management) and evaluated the performance of 77 fourth-year medical students. The skill and content domains of the OSCE were carefully defined, agreed upon, sampled and reproduced. This qualitative evaluation of the examination was both adequate and appropriate. We achieved both acceptable interstation and intertask reliability. When correlated with concurrent methods of evaluation we found the OSCE to be an accurate measure of paediatric knowledge and patient management skills. The OSCE did not correlate, however, with traditional measures of clinical skills including history-taking and physical examination. Our OSCE, as outlined, offers an objective means of identifying weaknesses and strengths in specific areas of clinical competence and is therefore an important addition to the traditional tools of evaluation.     

Standardized patients: a long-station clinical examination format

Summary. Data from the first 20 periods of a long-station clinical performance examination for a 4-week required clerkship in family medicine were examined in order to assess the reliability and validity of the examination. Data from 304 students were examined for station, case scenario and examiner effects and results compared to short-station formats. A significant examiner effect was found but there were no differences in student performance for station or case scenario. These findings reflect examiner specificity cited in the literature for short station examinations, but not case specificity. The source of variability for this examination appears to be primarily examiner effect. There was a significant correlation between student scores on the two cases, and raters tended to rank order students similarly in spite of variability in mean rater score. Scores on the CPE correlated with other measures of clinical performance as well as other methods of student evaluation for the clerkship providing some evidence for construct and criterion-related validity. CPE cases were developed from clerkship objectives but examination of the test blueprint revealed some gaps in the extent to which the CPE covers the course content. CPE developers are working to increase interrater reliability through examiner training and further standardize case scenarios through checklists and patient training. Additional cases are being developed to increase the content validity of the examination.     

Personality,clinical performance and knowledge in paediatric residents

Summary. This study examines the relationship between personality, knowledge and teachers' evaluations of paediatric residents at a large, urban teaching hospital. All residents (n= 30 ) were graduates of foreign medical schools. Each resident completed the Myers-Briggs Type Indicator (MBTI) as a measure of personality and was rated (on a scale of 1-7) by teachers on each of 35 items representing 14 different aspects of clinical performance. These data were correlated with performance on the American Board of Pediatrics In-training Examination (ABPITE), and with demographic data. Ratings of knowledge were directly associated with scores on the ABPITE (r= 0.51 , P < 0.01 ) and with the MBTI continuous score for extraversion (r= 0.51 , P < 0.01 ), but inversely associated with age (r=?0.41, P < 0.01 ). ABPITE scores were also associated directly with MBTI Extraversion (r= 0.44 , P < 0.01 ) and inversely with age (r=?0.56, P < 0.001 ). Age and MBTI Extraversion were independently significant predictors of ABPITE (Multiple R= 0.64, P < 0.01 ) and ratings of knowledge (Multiple R= 0.59, P < 0.01 ). Results suggest that teachers' evaluations of knowledge have validity, and that age and introversion/extroversion play a significant role in both subjective and objective evaluations of paediatric knowledge. Demographic and personality variables may be predictive of knowledge acquisition, but are unrelated to many dimensions of clinical performance.     

Medical student experience of London general practice teaching attachments

Forty-eight students kept a log diary of activities during their central London general practice teaching attachments associated with the Department of Primary Health Care of University College and Middlesex School of Medicine. The students each saw on average 96 patients per week, of whom 69% were discussed by the general practitioner with the student after the consultation. Students spent an average of 21.5 hours a week sitting in with the general practitioner. While most of this time was as a passive observer, the students were also able to participate more actively, personally taking histories for a median of 1.25 hours a week and personally examining patients for a median of 1.7 hours a week. During these periods of active involvement each student personally took a mean of 10 short and 2.5 long histories per week and performed a mean of 25.5 short and 1.2 long examinations per week. General practitioners to whom the students were attached spent a mean of 4 hours a week on (patient-oriented) teaching. The tuition was highly rated by the students in terms of both usefulness and stimulation. Students also received a mean of 2.3 hours a week of teaching from other members of the primary health care team, which was somewhat less well received. Areas for improvement were: the relatively few home visits (median of 6 per week) per student; the limited time students spent on self-education (average of 65 minutes per week); and the few practical procedures performed by the students. Students could also be encouraged to play a more active role in examining and interviewing patients.     

Focus group study of student physiotherapists’ perceptions of reflection

Context The reflective practice module in the physiotherapy programme at the University of Limerick, Ireland represents the first incidence of the inclusion of such a module within physiotherapy curricula in Ireland. However, research examining the contribution of reflection as a means of learning is limited, particularly from the student perspective. Objectives This study sought to explore students’ perceptions of reflection and its potential contribution to their development before and after the module. Methods A qualitative research methodology using focus groups was employed to evaluate physiotherapy undergraduate students’ perceptions of the module. Three focus groups were held in total. Two were held with Year 3 students, before and after their reflective practice module, respectively, to examine any changes in their perceptions of reflection. A third was held with Year 4 students to determine their perceptions after both the module and subsequent clinical placements. Sessions were audiotaped, transcribed and subjected to in‐depth thematic analysis to resolve the significant themes that emerged from the data. Results Students reported a more advanced level of reflective ability post‐module completion. They perceived personal and professional benefits to practising reflection and recognised these skills as strategies with which they could continue to facilitate their professional development. For students, time constraints in the clinical setting represented a barrier to reflection. Conclusions Students support inclusion of the module in their training, acknowledging its role in improving their confidence and clinical reasoning, and facilitating continuing professional development. Further studies are required to generalise these findings to a wider population.     

Changes in students' perceptions of clinical teaching as a result of general-practice clerkships

This study investigated whether a 3-week clerkship for third-year medical students in general-practice doctors' offices changed the students' perceptions of clinical teaching. The Preceptor Evaluation Questionnaire was administered to 138 junior medical students before and after their clerkships. The result of the study indicated that the clerkship had a positive effect on the students' perceptions. This finding was not related to their experiences in other clerkships nor to the grades they received for the clerkship.
The results indicate that the teaching relationship established between student and family doctor significantly affects students' perceptions of aspects of effective clinical teaching.     

Graduate evaluation of undergraduate teaching programmes in paediatrics

In a new undergraduate teaching course in child health, medical students were encouraged to prepare and present topics to their colleagues. These presentations covered major subjects in paediatric medicine and were supervised by an experienced teacher. Students were then asked to evaluate the programme at the end of their course in child health and after graduation. Evaluation scores for these presentations were high at the end of the course. Spontaneous comments suggested some disadvantages such as poor or too-detailed student presentations. However, some noted the value of personal research. After graduation, more positive comments were made and the majority felt that the course had advantages in the area of personal research, that it was better than or as good as other teaching methods, and that it should be continued in its present format.     

medline training for medical students integrated into the clinical curriculum†

This study was initiated in response to the availability of unlimited access to medline via the PaperChase interface at the host institution. The goal of the study was to analyse the usage of medline by medical students during their third-year clinical clerkship in paediatrics. This was achieved by first giving them a formal demonstration of medline and then longitudinally administering a structured questionnaire at several points during their paediatric clerkship. Since medical students were required to write a thesis for this clerkship, which included an analysis of the primary literature, there was an identified need for using the system. This study analyses the students' perceptions and searching activities after formal training and upon having unlimited access to medline .     

Teaching clinical methods to medical students

Summary. This paper demonstrates that it is feasible to teach clinical methods in general practice and describes the organization of an appropriate attachment. Willing practitioners, properly briefed, are competent to undertake clinical methods teaching and the attachment provides satisfaction to both students and teachers. It is possible to provide in primary care the elements which seem to be the key to this outcome — protected time, teaching in very small groups, direct observation and concentration on systematic examination rather than on particular physical signs. There is a strong case for the promoting of clinical methods teaching by general practitioners.     

Medical undergraduate teaching of paediatrics in the community

The undergraduate teaching of childhood illness in the community is described. This teaching includes a cross-section of acute childhood illness together with some contact with handicapped children in their own homes. Although there were only four teaching sessions, the frequency of conditions studied approximated to that in general practice     

Interdisciplinary teaching of community paediatrics

An experiment of combining decentralized and interdisciplinary teaching is described. Seminars were arranged at a district hospital for medical, nursing and social work students to discuss families with a chronically ill child. The purpose of these seminars was to teach co-operation between different health care workers, and to stress the importance of becoming acquainted with the family situation, and learn what a child's illness means to the whole family.
An evaluation of the first three seminars shows that this type of teaching was very well received by the students, professionals and families concerned. The students considered house calls an important part of their education and felt that more interdisciplinary teaching should be included in their curriculum. In addition to the teaching aspect, the families involved also considered the seminars a positive experience as they received more information about their child's disease, and deficiencies in treatment could be corrected.     

Effective teaching of communication skills for medical practice: selecting an appropriate clinical context

Increasingly, courses in communication skills are being incorporated into medical training. In order for communication skills to be effectively maintained in post-training medical practice, they must be taught within an appropriate clinical context. The present paper describes and provides rationale for seven criteria by which to select clinical issues which are appropriate foci for communication skills courses. The criteria are : (1) the issue must be one which is encountered frequently in clinical practice; (2) the issue must be associated with a high burden of illness; (3) there must be evidence that practitioners need to improve skills for dealing with the issue; (4) there must be an intervention, of which communication skills are an integral component, that is demonstrably effective for dealing with the clinical issue; (5) the intervention must represent a cost-effective means of dealing with the issue; (6) the intervention must be acceptable to doctors and be able to be incorporated into routine medical practice; (7) the intervention must be acceptable to patients. Examples of clinical issues which fit these criteria are given in the paper and include smoking, hazardous alcohol consumption, non-adherence to treatment instructions, overdue cervical screening, inappropriate diet, recovery from medical interventions, and breaking bad news to patients.     

Medical student and faculty perceptions of desirable primary care teaching site characteristics

BACKGROUND: As the need to expand and improve primary care teaching experiences has mushroomed, the need to identify desirable preceptor and site characteristics has also grown. PURPOSE: The current study was designed to assess the relative importance students and preceptors place on site versus preceptor characteristics. METHODS: After a required year-long primary care experience, third-year medical students (n=39) and primary care preceptors (n=20) completed a Q-sort exercise. RESULTS: In all analyses preceptor characteristics were ranked highest. The highest ranked item for students and preceptors was 'The preceptor allows student to assume increasing levels of responsibility'. Seven of the highest ranked items appear on both groups' top 10 list. CONCLUSIONS: There is a high level of agreement between students and preceptors that preceptor rather than site characteristics make the vital difference in primary care educational experiences. When students and preceptors disagree, it is because preceptors see their role- modelling characteristics related to relationships with patients as important, while students are more interested in the hands-on learning opportunities afforded by the preceptor.     

An early ‘clinical trial’ as a teaching exercise: the Book of Daniel 1.1–15 (1.1–20)

Summary. This article discusses a very early clinical trial from the Old Testament. One of Daniel's and his companions' tribulations in Babylonia is explicated within the framework of the modern clinical trial. Even if many, or maybe even most, guidelines for good clinical trial practice are violated (it can even be argued that this is not really a clinical trial), a discussion of this biblico-historical episode in, for example, a problem-based course in (clinical) epidemiology or a course in the critical appraisal of the literature, can be a useful (and possibly entertaining) exercise.     

An inventory to improve clinical teaching in the general internal medicine clinic

The increasing occurrence of outpatient medical care has led to the need for more and better medical education in the clinic. the Wisconsin Inventory of Clinic Teaching (WICT) was developed to improve the teaching of attending doctors in a general internal medicine clinic. The items on the inventory were derived from interviews with residents and attending doctors. The inventory was shown to have validity, and to be reliable with internal consistency correlations. We found an interesting disparity between attending doctors' and residents' expectations concerning clinical teaching. The instrument is in use as part of a teaching improvement programme for attending doctors in a general internal medicine clinic.     

The clinical learning environments in medicine, paediatrics and surgery clerkships

Using questionnaires, the students of the 1981 graduating class from McGill's Faculty of Medicine were investigated for their perceptions of the nature of the clinical instruction and of the roles of the consultant, resident and intern staffs during clerkships in medicine, paediatrics and surgery. Personal student diaries were used to assess time spent on various clinical activities. The results indicated that students perceived learning to be different in the three disciplines, with the acquisition of clinical skills (technical and problem-solving skills) greater in medicine and surgery than in paediatrics, and the acquisition of interpersonal skills and factual knowledge greater in paediatrics than in medicine and surgery. Students perceived themselves as passive observers in surgery and formed personal relationships more easily with staff in medicine and paediatrics than in surgery. In contrast, students perceived more emphasis on team effort in surgery. Time spent on activities related to direct patient encounter was greatest in medicine. The findings suggest that the learning environments in clinical disciplines are not homogeneous and this has implications for curricular planning and clinical teaching.     

A factor analytic study of clinical competence in undergraduate psychiatry

This study used factor analysis to define the components of clinical competence of medical students during their undergraduate psychiatric training. Four factors were defined; factor 1 related to cognitive and psychological problem-solving; factor 2 tapped the interpersonal and observational skills students showed with patients; factor 3 was characterized by knowledge in the examination setting, and factor 4 related to students' capacity to demonstrate their ability in an interpersonal setting. These are similar to the component skills of clinical competence demonstrated by students in other areas of the medical curriculum. They also correspond to the skills which Walton (1986) has suggested should be focused upon in undergraduate psychiatric education.     

The development and evaluation of a method of teaching liaison psychiatry to medical students

A method is described of teaching medical students aspects of psychiatry related to their work with physically ill patients. The teaching requires few resources and focuses on a limited number of educational objectives centred on the acquisition of knowledge of psychiatry relevant to medical practice. The course comprised eight weekly one-hour seminars held during the first clinical year attachment to medical firms, prior to the students' psychiatry attachments. Clinically based, traditional 'bedside' teaching was used, involving patients known to the students. In a preliminary analysis of the effects of the teaching, students who had participated in the seminars scored significantly better on a test of liaison psychiatry knowledge than a matched group who had not experienced the course.     

An audit of clinical teaching: an approach to one performance indicator of educational competence

This paper describes a simple, self-administered audit of the year-by-year clinical examination results of 149 students taught by the writer (Dr X) over a 9-year tenure (1977-1985) as a Senior Lecturer in Surgery at the University of Western Australia. The clinical examination results of these 149 students are compared with the clinical examination results of a total peer group cohort of 1567 students taught by the writer's colleagues in the same academic department over the same years. The limitations and applicability of this method of audit are discussed, and it is argued that this type of self-assessment might be of use, in conjunction with other evidence, in the documentation of teaching expertise. Such documentation might be of use in applications for clinical teaching appointments or promotions.