Effects of communication skills training on students'' diagnostic efficiency

This study evaluates the impact of a training programme in communications skills on subsequent diagnostic efficiency. Videotaped history-taking interviews conducted by groups of specially trained and control groups of students were rated for their diagnostic efficiency by two medical practitioners. Students in the trained group had shown greatly increased skills in interviewing and interpersonal effectiveness as a result of their training. A comparison of ratings given by the two experimentally naive, independent observers revealed that trained students were significantly better at eliciting full, relevant data from patients--they were diagnostically more efficient, but took no longer than their control group counterparts to elicit the information. Further research with the medical interview rating scale will clarify the skills required of medical students in interviewing and diagnosis and facilitate remedial training for students who show poor interview skills.     

The passing score in the Objective Structured Clinical Examination

The objective structured clinical examination (OSCE) now has an established place in the assessment of the medical undergraduate. While much has been written about the reliability of the OSCE, empirical work on the determination of the passing score which represents competence on the OSCE is rarely encountered. If the OSCE is to play its role in the 'high stakes' testing of clinical competence, it is important that this passing score be set reliably and defensibly. This article illustrates how a two-session modified Angoff standard-setting procedure is used to set the passing score on a 14 station Obstetrics and Gynaecology OSCE used to assess final year students at The Queen's University of Belfast. The Angoff methodology harnesses the professional judgement of expert judges to establish defensible standards. Four university teachers, five non-academic consultants and six junior clinical staff took part in a two-session Angoff standard-setting procedure. In the first session, the judges (individually and in silence) used their professional judgement to estimate the score which a minimally competent final year obstetrics and gynaecology student should achieve on each tested element of the OSCE. In the second session they revised their session 1 judgements in the light of the OSCE scores of real students and the opportunity for structured discussion. The passing score for the OSCE is reported together with the statistical measures which assure its reliability.     

The effects of the introduction of a system of mandatory formative assessment for general practice trainees

A series of questionnaires was used to determine the amount of formative assessment going on within the West of Scotland Region of the UK and its perceived value by trainees (General Practice [GP] Registrars) both at the beginning and end of training. The first survey was carried out in 1989 and the second in 1994. Trainers and course organizers were surveyed in 1994 to determine their attitudes to the use of videotaped consultations for formative assessment. In 1991, the region had initiated a mandatory formative assessment programme which included regular use of videotaped consultations, confidence rating scales and Manchester ratings (RCGP Occasional Paper Number 40). The use of a range of assessment methods for formative assessment of GP Registrars increased considerably between 1989 and 1994. The percentage of trainees using videotaped consultation analysis increased from 76 to 94%, for Manchester ratings from 52 to 68% and for confidence rating scales from 63 to 74%. Video and confidence scales were rated highly by trainees who were assessed by them and by most trainees at the start of the year, but less highly by those who had not been assessed by them by the end of the trainee year. Manchester ratings were not thought to be as useful, and for these there was no difference between users and non-users. Despite the mandatory system, a significant number of trainees (76%) were still not receiving the minimum assessment stipulated. Trainers rated lack of time as the main limiting factor to the greater use of video. We conclude that trainees who are exposed to assessment methods, particularly video, find it useful, but that some trainees are still receiving less than is optimal. We propose increased trainer education and intensified monitoring of the assessment carried out in training Practices.     

Preliminary evaluation of "interpreter" role plays in teaching communication skills to medical undergraduates

RATIONALE AND OBJECTIVES: Multiculturalism presents linguistic obstacles to health care provision. We explored the early introduction of "interpreter" role-play exercises in teaching medical undergraduates communication skills. The interpreter role creates a natural barrier in communication providing an active prompt for recognizing learning needs in this area. METHODS: Bilingual Cantonese first-year medical students (n=160) were randomly allocated to either "Observer" or "Interpreter" role plays at a small-group introductory communication skills workshop using a quasi experimental design, counterbalanced across tutors. Students assessed their own skill competence before and, together with their perceptions of the different role plays' effectiveness, again after the workshop, using an anonymous 16 item Likert-type scale, analysed using ANOVA and MANOVA. RESULTS: Students' assessments of their skills improved significantly following the workshop (F=73.19 [1,156], P=0.0009). Students in the observer group reported greater changes in their scores following the workshop than did students in the interpreter group (F=4.84 [1,156], P=0.029), largely due to improvement in perceived skill (F=4.38 [1,156], P=0.038) rather than perceived programme effectiveness (F=3.13 [1,156], P > 0.05). Subsequent MANOVA indicated no main effect of observer/interpreter conditions, indicating these differences could be attributed to chance alone (F=1.41 [16 141], P > 0.05). CONCLUSION: The workshop positively influenced students' perceived communication skills, but the "Interpreter" role was less effective than the "Observer" role in achieving this. Future studies should examine whether interpreter role plays introduced later in the medical programme are beneficial.     

Predicting communication skills with a paper-and-pencil test

AIM: This study was conducted to investigate the value of a written knowledge test of communication skills for predicting scores on a performance test of communication skills. METHOD: A paper-and-pencil test of knowledge about communication skills and a performance test of communication skills, consisting of four stations with standardised patients, were administered to students of two classes of the medical schools of Maastricht and Leiden, the Netherlands. The results on these tests were compared. RESULTS: From the results of both instruments, the classes of the participating students could be recognised equally well: 60% correct qualifications of the classes by the knowledge test and 64% by the multiple station examination. Between the two tests an overall, disattenuated correlation of 0.60 was found (N=133, P < 0.01), suggesting moderate predictive value of the knowledge test for the performance test of communication skills. The correlation is stronger for students from Maastricht medical school than for their colleagues in Leiden. Correlation between the knowledge of communication skills test and other available test results of the participating Maastricht students is close to zero, suggesting that the test measures a distinct quality of students' competence. DISCUSSION: The paper-and-pencil test of knowledge of communication skills has predictive value for the performance of these skills, but this value seems to be less pronounced than similar findings for clinical procedural skills. The stronger relationship between 'knowing how' and 'showing' in the Maastricht student group might be indicative of an effect of the training format.     

Providing feedback to students on clinical skills by using the Objective Structured Clinical Examination

Student opinions of some methods of how and when to provide feedback on clinical skills in the Objective Structured Clinical Examination (OSCE) were gathered. Feedback with their marked checklists plus a model videorecording was preferred to either on its own. Feedback was preferred during the examination rather than after it. No undue stress appeared to be associated with receiving feedback during the examination. Valuable extra instruction to students can be provided with minimal staff input.     

Undergraduate medical students' views about a reflective portfolio assessment of their communication skills learning

INTRODUCTION: To date, no studies have examined preclinical medical students' views about portfolios. Since portfolios are becoming increasingly valued in medical education, this study explores second-year medical students' views about a reflective portfolio assessment of their communication skills. METHODS: 178 second-year medical students at the University of Nottingham completed the 18-item reflective portfolio questionnaire (RPQ) (alpha = 0.716) and a personal details questionnaire three days before submitting their portfolio assessment for communication skills. Data were analysed using univariate and multivariate statistics on SPSS Version 10.0. RESULTS: Total scores on the RPQ ranged from 40 to 75 (mean 58.28, SD 7.08). Significant relationships existed between RPQ total scores and students' ratings of their reflection skills (rs = 0.322, P < 0.001), RPQ total scores and students' confidence building another portfolio (T = 4.381, d.f. = 176, P < 0.001), and RPQ total scores and students' marks for their reflective portfolio assessment (rs = 0.167, P = 0.029). Students with more positive views about reflective portfolios were more likely to rate their reflection skills as good, receive better marks for their portfolio assessment, and be more confident building another portfolio. DISCUSSION: This study begins to highlight preclinical medical students' views about reflective portfolios. However, further research is required using qualitative studies to explore students' views in depth. Medical educators should be encouraged to consider introducing portfolios as a method of formative and summative assessment earlier in the medical curriculum.     

The effectiveness of immediate feedback during the Objective Structured Clinical Examination

Using eight different physical examination or technical stations, 400 examinations were conducted to evaluate the effectiveness of immediate feedback during the Objective Structured Clinical Examination (OSCE). The test group comprised 50 medical students who underwent a standard 4-minute examination followed by 2 minutes of feedback. Immediately following feedback the students repeated an identical 4-minute examination scored by the same examiners. The control group consisted of 50 students from the same class who underwent an identical testing sequence, but instead of receiving feedback, they were instructed to continue their examinations for an additional 2 minutes before repeating the stations. Simple repetition of the task did not significantly improve score (mean increase 2.0%, NS). Extending the testing period from 4 to 6 minutes resulted in a small but significant increase in score (mean 6.7%, P less than 0.001). However, there was a much larger increase in the scores obtained following 2 minutes of immediate feedback compared to pre-feedback performance (mean 26.3%, P less than 0.0001). The majority of students and examiners felt that feedback, as administered in this study, was valuable both as a learning and teaching experience. Short periods of immediate feedback during an OSCE are practical and can improve competency in the performance of criterion-based tasks, at least over the short term. In addition, such feedback provides students with valuable self-assessment that may stimulate further learning.     

The lessons of deafness: deafness awareness and communication skills training with medical students

This paper describes a 1-day course on deafness awareness and communication skills with fourth-year medical undergraduates and summarizes their evaluation of the course. Deaf people commonly experience major communication difficulties with doctors. This course gives students an awareness of deafness (and of the insights deaf people can give to an understanding of communication)--the lessons of deafness--and challenges some critical assumptions in medical practice. Students are given personal experience of deafness and of the consequent powerlessness and loss of self-confidence, while also learning new communication skills which they then apply in a role-play consultation with a deaf patient. Systematic evaluation has been built into the programme and student opinion has been unanimously enthusiastic.     

The effect of a Structured Question Grid on the validity and perceived fairness of a medical long case assessment

PROBLEM: A perception that the reliability of our oral assessments of clinical competence was vitiated by lack of consistency in questioning. DESIGN: Parallel group controlled trial of a Structured Question Grid for use in clinical assessments. The Structured Question Grid required assessors to see the patient personally in advance of the student and to write down for each case the points they wished to examine. The Structured Question Grid limited assessors to two questions on each point, one designated a pass question and one at a higher level. Three basic science and three clinical reasoning issues were required, so that a total of 12 questions was allowed. SETTING: Small (70 students/year) undergraduate medical school with an integrated, problem-based curriculum. SUBJECTS: Sixty-seven students in the fourth year of a 5-year course were assessed, each seeing one patient and being examined by a pair of assessors. Assessor pairs were allocated to use the Structured Question Grid or to assess according to their usual practice. RESULTS: After the assessment but before being informed of the result the students completed a questionnaire on their experience and gave their performance a score between 0 and 100. The questions asked were based on focus group discussions with a previous student cohort, and concerned principally the perceived fairness and subjective validity of the assessment. The assessors independently completed a similar questionnaire, gave the student's performance a score between 0 and 100, and assigned an overall pass/fail grade. CONCLUSIONS: No difference was detected between students' or assessors' views of the fairness of the assessment for assessors who had used the Structured Question Grid compared to those who had not. Students whose assessors used the Structured Question Grid considered the assessment less representative of their ability. No difference was detected in the chance of students being assessed as failing or on the likelihood of a discrepancy between students' and assessors' ratings of students as passing or failing.     

Teaching clinical skills to medical students: the use of simulated patients and videotaping in general practice

This paper describes the use of simulated patients in a problem-based teaching programme designed to develop the clinical skills of medical students during a 5-week course in general practice in the University of Leicester. The logistics of training and the advantages of using simulated patients are discussed. Simulated patients are practicable and acceptable in a United Kingdom medical school, and the reaction of students to their use is favourable.     

Use of an innovative video feedback technique to enhance communication skills training

CONTEXT: Despite growing interest in medical communication by certification bodies, significant methodological and logistic challenges are evident in experiential methods of instruction. OBJECTIVE: There were three study objectives: 1) to explore the acceptability of an innovative video feedback programme to residents and faculty; 2) to evaluate a brief teaching intervention comprising the video feedback innovation when linked to a one-hour didactic and role-play teaching session on paediatric residents' communication with a simulated patient; and 3) to explore the impact of resident gender on communication change. DESIGN: Pre/post comparison of residents' performance in videotaped interviews with simulated patients before and after the teaching intervention. Individually tailored feedback on targeted communication skills was facilitated by embedding the Roter Interaction Analysis System (RIAS) within a software platform that presents a fully coded interview with instant search and review features. SETTING/PARTICIPANTS: 28 first year residents in a large, urban, paediatric residency programme. RESULTS: Communication changes following the teaching intervention were demonstrated through significant improvements in residents' performance with simulated patients pre and post teaching and feedback. Using paired t-tests, differences include: reduced verbal dominance; increased use of open-ended questions; increased use of empathy; and increased partnership building and problem solving for therapeutic regimen adherence. Female residents demonstrated greater communication change than males. CONCLUSIONS: The RIAS embedded CD-ROM provides a flexible structure for individually tailoring feedback of targeted communication skills that is effective in facilitating communication change as part of a very brief teaching intervention.     

The reliability of assessment criteria for undergraduate medical students' communication skills portfolios: the Nottingham experience

INTRODUCTION: Some educators have argued that portfolios should not be assessed summatively because there is little evidence supporting the reliability of their assessment. This study aims to determine the reliability of assessment criteria used for a portfolio at the University of Nottingham. METHODS: Two independent analysts assessed a random sample of portfolios (n = 100, 49.5%) using criterion-referenced assessment. Students' performances were examined against subjective items in five areas: 1) portfolio structure, 2) level of critical reflection, 3) level of skills development, 4) use of documentary evidence, and 5) use of relevant literature. These subjective judgements were later converted into quantitative scales ranging from 0 to 3 so that interrater reliability could be established. The level of agreement between the two analysts for the total percentage score was established using an intraclass correlation coefficient and for the individual items using weighted kappa coefficients. RESULTS: The level of agreement between the two raters for the total percentage score was 0.771 (95% CI = 0.678, 0.840), as measured by an intraclass correlation coefficient. The levels of agreement between the two raters for the individual items of the assessment criteria ranged from kappa=0.359 (item 3) to kappa=0.693 (item 4). DISCUSSION: This study provides some support for the summative assessment of portfolios. The findings suggest that discussion and negotiation between independent assessors can enhance the reliability of assessment criteria. Therefore, medical educators are encouraged to use such procedures in the summative assessment of portfolios.     

Orthopaedic undergraduate assessment

Summary. The multiple choice, viva voce and consultant marks from 327 students in the Orthopaedic Undergraduate Course at the University of Bristol were compared. The assessments were based on 84 questions in the multiple choice questionnaire, 30 questions in the viva voce and four questions in the consultant assessment. The consultant assessment correlated least well with the other two marks. It is suggested that in short courses consultant grades be supplemented by objective examinations.     

Formative assessment of the consultation performance of medical students in the setting of general practice using a modified version of the leicester assessment package

OBJECTIVE: To evaluate the use of a modified version of the Leicester Assessment Package (LAP) in the formative assessment of the consultation performance of medical students with particular reference to validity, inter-assessor reliability, acceptability, feasibility and educational impact. DESIGN: 180 third and fourth year Leicester medical students were directly observed consulting with six general practice patients and independently assessed by a pair of assessors. A total of 70 practice and 16 departmental assessors took part. Performance scores were subjected to generalizability analysis and students' views of the assessment were gathered by questionnaire. RESULTS: Four of the five categories of consultation performance (Interviewing and history taking, Patient management, Problem solving and Behaviour and relationship with patients) were assessed in over 99% of consultations and Physical examination was assessed in 94%. Seventy-six percent of assessors reported that the case mix was 'satisfactory' and 20% that it was 'borderline'; 85% of students believed it to have been satisfactory. Generalizability analysis indicates that two independent assessors assessing the performance of students across six consultations would achieve a reliability of 0.94 in making pass or fail decisions. Ninety-eight percent of students perceived that their particular strengths and weaknesses were correctly identified, 99% that they were given specific advice on how to improve their performance and 98% believed that the feedback they had received would have long-term benefit. CONCLUSIONS: The modified version of the LAP is valid, reliable and feasible in formative assessment of the consultation performance of medical students. Furthermore, almost all students found the process fair and believed it was likely to lead to improvements in their consultation performance. This approach may also be applicable to regulatory assessment as it accurately identifies students at the pass/fail margin.     

Teaching the teachers — a needs assessment of tutors for a new clinical skills course

There is an increasing trend in undergraduate education towards teaching clinical skills from a community base. A new clinical curriculum was introduced in Newcastle upon Tyne in 1995, beginning with an integrated clinical skills course. Although the attitudes and views of general practitioners (GPs) towards community-based clinical teaching have previously been reported, their perceived training needs have not been formally identified. The aims of this study were to identify the competencies needed by GPs for community-based clinical skills teaching, to compare and contrast these needs with their hospital colleagues, and to use the results to develop a teaching programme for the clinical tutors involved in the new course. In order of priority, the GPs and hospital tutors expressed similar needs: small-group teaching skills, assessing student needs, giving effective feedback and assessment of student performance, with a preference for the teaching to be organized within local teaching units. Most GPs and hospital tutors (73 and 69%, respectively) requested a distance-learning pack to complement the teaching. General practitioners rated resources for improving their individual clinical skills more highly than their hospital colleagues: for example, videotapes demonstrating examination techniques. Forty-six per cent of GP tutors had received some formal training in teaching methods compared to 29% of hospital tutors. The implications of the results for developing a 'Teaching the Teachers' course for clinical tutors are discussed.     

A needs-based study and examination skills course improves students' performance

BACKGROUND: Adult learning theory suggests that learning is most effective when related to need, when driven by the learner and when it is flexible. We describe the effect of an educational intervention that was driven by student need, and largely designed by students. METHODS: We undertook a needs assessment of fifth year medical students' study needs. Based on this, we helped them design a course to meet these needs. This was predominantly related to study skills and a practice objective structured clinical examination (OSCE). We evaluated the course by asking for student opinion and by measuring the effect on student performance in a high stakes medical school examination (written examination and OSCE). FINDINGS: Despite the course being run voluntarily and in after-hours sessions, 80-90% of the medical student class attended each session. Student performance on the end of year examinations was significantly enhanced in the year of the intervention, compared with previous years and with students from other schools sitting identical examinations in the same year. INTERPRETATION: Learning activities that are directly based on student needs, that focus on study and examination techniques, and that are largely student-driven, result in effective and valuable outcomes.