Evaluating surgeons' informed decision making skills: pilot test using a videoconferenced standardised patient

BACKGROUND: Standardised patients (SPs) are effective in evaluating communication skills, but not every training site may have the resources to develop and maintain SP programmes. OBJECTIVES: To test whether videoconferencing technology (VT) could enable an interaction between an SP and an orthopaedic surgeon that would allow the SP to accurately evaluate the surgeon's informed decision making (IDM) skills. We also assessed whether this sort of interaction was acceptable to orthopaedic surgeons as a means of learning IDM skills. METHODS: We trained an SP to represent a 75-year-old woman considering hip replacement surgery. Orthopaedic surgeons in Chicago individually consulted with the SP in Philadelphia; each participant could see and hear the other on large television screens. The SP evaluated the surgeons' advice using a 23-item checklist of IDM elements, and gave each surgeon verbal and written feedback on his IDM skills. The surgeons then gave their evaluations of the exercise. RESULTS: Twenty-two surgeons completed the project. The SP was > or = 80% accurate in classifying 20 of the 23 IDM skills when compared to a clinician rater. Although 12 (55%) of the orthopaedic surgeons felt that some aspects of the technology were distracting, most were pleased with it, and 19 of 22 (86%) would recommend the videoconferenced SP interaction to their colleagues as a means of learning IDM skills. CONCLUSIONS: These results suggest that VT allows accurate evaluation of IDM skills in a format that is acceptable to orthopaedic surgeons. Videoconferencing technology may be useful in long-distance SP communication assessment for a variety of learners.     

Impact of a new curriculum on medical students' interpersonal and interviewing skills

OBJECTIVES: In 1998 we reported on the rise and fall of medical student communication skills during the 4 years of medical school. Since then, the University of Connecticut School of Medicine has completed a major curriculum renewal project with an emphasis on early clinical work, lifelong learning and more ambulatory training. The goals of this study were to compare students' interviewing and interpersonal skills in standardised patient (SP) assessments in the old and new curricula and to assess the success of the new curriculum in preventing a decline in student skills in this domain. METHODS: The clinical skills of 202 students were measured longitudinally during encounters with SPs in each of their 4 years of medical school. Students in this study and the earlier study were evaluated using the Arizona Clinical Interviewing Rating (ACIR) Scale. RESULTS: Compared with students from the previous curriculum, students on the new curriculum in this study showed an improvement in ACIR scores. Year 1 mean ACIR scores (1 = poor to 5 = excellent) were, respectively, 3.6 for the old curriculum cohort and 4.0 for the new curriculum group. In Year 4 the mean score for the old curriculum cohort was 3.7 and that for the new curriculum group was 3.8. Students on the new curriculum still showed a decline in ACIR scores from Years 1 to 4, but it was not as severe a decline as it had been previously. CONCLUSIONS: Pre-clinical medical students perform better on measures of interpersonal communication than their clinical counterparts. The students who participated in the new curriculum demonstrated an earlier acquisition of and a less steep decline in interviewing and interpersonal skills during the course of medical school.     

Medical students' perceptions of identity in communication skills training: a qualitative study

Objectives  Communication skills training in undergraduate medical education is considered to play an important role in medical students' formation of their professional identity. This qualitative study explores Year 1 students' perceptions of their identities when practising communication skills with real patients.
Methods  A total of 23 individual semi-structured interviews and two focus group discussions were conducted with 10 students during their first year of communication skills training. All interviews and discussions were audio-recorded, transcribed and analysed for emergent themes relating to identity.
Results  Students struggled to communicate professionally with patients because of a lack of clinical knowledge and skills. Consequently, students enacted other identities, yet patients perceived them differently, causing conversational ambiguities.
Discussion  Students' perceptions challenge educational goals, suggesting that there is limited potential for the formation of professional identity through early training. Teacher-doctors must acknowledge how students' low levels of clinical competence and patients' behaviour complicate students' identity formation.     

Simulating the longitudinal doctor-patient relationship: experiences of simulated patients in successive consultations

CONTEXT: The use of simulated patients (SPs) in teaching communication and practical skills at medical schools is mostly limited to single-case use: a student has 1 consultation with an SP and receives feedback afterwards. Very little literature is available that describes consecutive consultations between the same student and SP. In this study, we explored the experiences of SPs in a new, longitudinal SP programme in which SPs met the same Year 3 students ('GP') in 4 consecutive consultations during the year. The SPs suffered from a chronic disease in their patient roles. METHODS: Four focus group discussions were conducted with 23 SPs (8 men, 15 women; average age 60.9 years) who had performed in the new programme. Discussions were semi-structured and followed a pre-established interview guide. Data were categorised by 3 independent raters. RESULTS: The SPs described the development of a more familiar relationship with students under the new programme, compared with single-case consultations. They developed specific expectations of students' performances. The SPs enjoyed participating in the programme and felt it was more realistic than single-case consultations. Feedback changed and became more detailed as SPs were able to compare consultations; students' response to feedback could be experienced during the next consultation. DISCUSSION: Practising the development of a realistic, longitudinal doctor-patient relationship may help prepare students for real practice. Longitudinal feedback is now possible; it may be of higher quality and of benefit to SPs as well. These findings suggest new possibilities for SP-based education and research. Future studies should focus on quantitative analysis and students' perspectives.     

Assessment of patient management skills and clinical skills of practising doctors using computer-based case simulations and standardised patients

CONTEXT: Standardised assessments of practising doctors are receiving growing support, but theoretical and logistical issues pose serious obstacles. OBJECTIVES: To obtain reference performance levels from experienced doctors on computer-based case simulation (CCS) and standardised patient-based (SP) methods, and to evaluate the utility of these methods in diagnostic assessment. SETTING AND PARTICIPANTS: The study was carried out at a military tertiary care facility and involved 54 residents and credentialed staff from the emergency medicine, general surgery and internal medicine departments. MAIN OUTCOME MEASURES: Doctors completed 8 CCS and 8 SP cases targeted at doctors entering the profession. Standardised patient performances were compared to archived Year 4 medical student data. RESULTS: While staff doctors and residents performed well on both CCS and SP cases, a wide range of scores was exhibited on all cases. There were no significant differences between the scores of participants from differing specialties or of varying experience. Among participants who completed both CCS and SP testing (n = 44), a moderate positive correlation between CCS and SP checklist scores was observed. There was a negative correlation between doctor experience and SP checklist scores. Whereas the time students spent with SPs varied little with clinical task, doctors appeared to spend more time on communication/counselling cases than on cases involving acute/chronic medical problems. CONCLUSION: Computer-based case simulations and standardised patient-based assessments may be useful as part of a multimodal programme to evaluate practising doctors. Additional study is needed on SP standard setting and scoring methods. Establishing empirical likelihoods for a range of performances on assessments of this character should receive priority.     

Acquisition of communication skills in postgraduate training for general practice

PURPOSE: The evidence suggests that a longitudinal training of communication skills embedded in a rich clinical context is most effective. In this study we evaluated the acquisition of communication skills under such conditions. METHODS: In a longitudinal design the communication skills of a randomly selected sample of 25 trainees of a three-year postgraduate training programme for general practice were assessed at the start and at the end of training. Eight videotaped real life consultations were rated per measurement and per trainee, using the MAAS-Global scoring list. The results were compared with each other and with those of a reference group of 94 experienced GPs. RESULTS: The mean score of the MAAS-Global was slightly increased at the end of training (2.4) compared with the start (2.2). No significant difference was found between the final results of the trainees and the reference group. According to the criteria of the rating scale the performance of both trainees and GPs was unsatisfactory. CONCLUSION: The results of this study indicate that communication skills do not improve in a three-year postgraduate training comprising both a rich clinical context and a longitudinal training of communication skills, and that an unsatisfactory level still exists at the end of training. Moreover, GPs do not acquire communication skills during independent practice as they perform comparably to the trainees. Further research into the measurement of communication skills, the teaching procedures, the role of the GP-trainer as a model and the influence of rotations through hospitals and the like, is required.     

Using standardised students in faculty development workshops to improve clinical teaching skills

PURPOSE: We describe the use of standardised students (SSs) in interdisciplinary faculty development programmes to improve clinical teaching skills. Standardised students are actual health professions students who are trained to portray a prototypical teaching challenge consistently across many encounters with different faculty participants. METHODS: The faculty development programmes described focused on the skills of providing feedback and brief clinical teaching. At the beginning of each session, each participant was videotaped in encounters with 2 different SSs. Using microteaching (an instructional method in which learners view short segments of their own videotaped performance and discuss the tapes with a facilitator, consultant or other workshop participants), each group of participants and instructors reviewed the tapes and reflected on the encounters, providing immediate feedback to participants and modelling different approaches to the same teaching problem. The same process was repeated with more complicated scenarios after 2 weeks and again after 6 months offering reinforcement, further practice and more sophisticated development of the strategies learned. Participants completed post-session evaluations and a follow-up telephone survey. RESULTS: A total of 36 faculty members from the colleges of medicine, dentistry, pharmacy and nursing participated in workshops in 2000-01. The workshops were rated as highly relevant to participants' teaching, and most participants reported that they had learned a great deal. Participants most appreciated reviewing the videotaped interactions, the feedback they received, the interactions with their colleagues, the interdisciplinary nature of the groups and the practical focus of the workshops. CONCLUSIONS: Standardised students provide a high fidelity, low risk, simulated environment in which faculty can reflect on and experiment with new teaching behaviours. Such encounters can enhance the effectiveness and impact of faculty development programmes to improve clinical teaching skills.     

Personality differences between doctors and their patients: implications for the teaching of communication skills

OBJECTIVE: To establish, as part of a wider study into specialty choice and job satisfaction, whether the personality profiles of a sample of doctors differed from those of the UK population at large, i.e. their potential patients, and the implications this might have for the doctor/patient consultation process. DESIGN: The Myers-Briggs Type Indicator (MBTI), which measures normal personality differences, was administered by post to five cohorts of doctors (n = 464) who had qualified from a London medical school during the 1980s. SETTING: United Kingdom. PARTICIPANTS: 313 (67.5%) of the medical graduates. MAIN OUTCOME MEASURES: Personality profiles of the respondents compared to those of the UK adult population norms, a proxy for their potential patients. RESULTS: The doctors in this sample differed significantly from the UK adult population norms on most of the dimensions of personality measured, including those which measure an individual's preferred mode of perception, i.e. how one likes to take in information and learn about things. This suggests potential points for miscommunication in the doctor/patient consultation process. CONCLUSIONS: This research should be replicated to see if the results are generalizable. Nevertheless, the findings do indicate that these doctors might benefit from education in the concept of psychological type differences and how these could affect communication with their patients. Training in how to "flex" their consultation style, when necessary, to take into account possible personality differences between themselves and their patients could enhance the outcome of the interaction for both parties.     

How clinical communication has become a core part of medical education in the UK

Context  This paper sets out to analyse and interpret the complex events of the last 20 years in order to understand how the teaching and learning of clinical communication has emerged as a core part of the modern undergraduate medical curriculum in most medical schools in the UK.
Methods  The paper analyses the effects of key political, sociological, historical and policy influences on clinical communication development.
Results  Political influences include: the effects of neo-liberalism on society and on the professions in general; the challenging of traditional notions of professionalism in medicine; the creation of an internal market within the National Health Service, and the disempowerment of the medical lobby. Sociological influences include: the effects of a 'marketised' society on medicine and subtle shifts in the doctor−patient relationship because of this; the emergence of globalised information through the Internet, and the influence of increased litigation against doctors. Historical influences include: the effects of a change in emphasis for medical education away from an inflated factual curriculum towards a curriculum that recognises the importance of student attitudes and the teaching and learning of clinical communication skills. Policy influences include the important effects of Tomorrow's Doctors and the Dearing Report on the modern medical curriculum.
Conclusions  The paper concludes with a developmental map that charts the complex influences on clinical communication teaching and learning and a brief commentary on the growing body of teachers who deliver and develop the subject today.     

Comparison of two instruments for assessing communication skills in a general practice objective structured clinical examination

OBJECTIVE: In recent decades, there has been increased interest in tools for assessing and improving the communication skills of general practice trainees. Recently, experts in the field rated the older Maas Global (MG) and the newer Common Ground (CG) instruments among the better communication skills assessment tools. This report seeks to establish their cross-validity. METHODS: Eighty trainees were observed by 2 raters for each instrument in 2 standardised patient stations from the final year objective structured clinical examination for Belgian trainee general practitioners. Each instrument was assigned 6 raters. RESULTS: Trainees showed the lowest mean scores for evaluating the consultation (MG7), summarising (MG11), addressing emotions (MG9) and addressing feelings (CG5). Inter-rater kappa statistics revealed fair-to-moderate agreement for the MG and slight-to-fair agreement for the CG. Cronbach's alpha was 0.78 for the MG and 0.89 for the CG. A generalisability study was only feasible for the MG: it was more helpful to increase the number of cases than the number of raters. Agreement between the instruments was examined using kappa statistics, Bland-Altman plots and multi-level analysis. Ranking the trainees for each instrument revealed similar results for the least competent trainees. Variances between and within trainees differed between instruments, whereas case specificity was comparable. Multi-level analysis also revealed a rater-item interaction effect. CONCLUSIONS: The 2 instruments have convergent validity, but the drawbacks of the CG, which has fewer items to be scored, include lower inter-rater reliability and score variance within trainees.     

Effects of communication skills training on students'' diagnostic efficiency

This study evaluates the impact of a training programme in communications skills on subsequent diagnostic efficiency. Videotaped history-taking interviews conducted by groups of specially trained and control groups of students were rated for their diagnostic efficiency by two medical practitioners. Students in the trained group had shown greatly increased skills in interviewing and interpersonal effectiveness as a result of their training. A comparison of ratings given by the two experimentally naive, independent observers revealed that trained students were significantly better at eliciting full, relevant data from patients--they were diagnostically more efficient, but took no longer than their control group counterparts to elicit the information. Further research with the medical interview rating scale will clarify the skills required of medical students in interviewing and diagnosis and facilitate remedial training for students who show poor interview skills.     

Evaluation of procedural skills training in an undergraduate curriculum

CONTEXT: A substantial proportion of medical students enter their intern year without any basic skills experience. Lack of experience is a significant source of stress for many junior doctors. OBJECTIVES: To evaluate the effect of a basic procedural skills tutorial for Year 3 medical students on their competence in relevant skills at Year 5. SUBJECTS: The control group consisted of 93 medical students who completed Year 3 in 1996. The intervention group consisted of 92 medical students who completed Year 3 in 1997. The intervention group received a practical skills tutorial in Year 3; the control group did not. Both groups were assessed on their practical skills competence during Year 5. METHODS: A 3-hour practical tutorial on injection and suturing techniques was delivered to the intervention group. The effectiveness of the intervention was assessed by self-reported experience of giving injections, inserting sutures and sustaining needlestick injuries, and by teacher-rated competency in four basic procedural skills. RESULTS: Students who received the Year 3 tutorial were significantly more likely to record a satisfactory assessment for their performance in all four basic skills compared with students who did not receive the tutorial. They were less likely than controls to refuse invitations to give injections, but not invitations to insert a suture, during Years 4 and 5. CONCLUSIONS: A single session of formalised teaching in procedural skills in the early stages of a medical degree can have long-term effectiveness in basic skills competence and may increase students' confidence to practise their skills.     

Conversation analysis, doctor-patient interaction and medical communication

INTRODUCTION: This paper introduces medical educators to the field of conversation analysis (CA) and its contributions to the understanding of the doctor-patient relationship. THE CONVERSATION ANALYSIS APPROACH: Conversation analysis attempts to build bridges both to the ethnographic and the coding and quantitative studies of medical interviews, but examines the medical interview as an arena of naturally occurring interaction. This implies distinctive orientations and issues regarding the analysis of doctor-patient interaction. We discuss the CA approach by highlighting 5 basic features that are important to the enterprise, briefly illustrating each issue with a point from research on the medical interview. These features of conversation analytic theory and method imply a systematic approach to the organisation in interaction that distinguishes it from studies that rely on anecdote, ethnographic inquiry or the systematic coding of utterances. CONVERSATION ANALYSIS AND THE MEDICAL INTERVIEW: We then highlight recent CA studies of the "phases" of the internal medicine clinic and the implications of these studies for medical education. We conclude with suggestions for how to incorporate CA into the medical curriculum. It fits with biopsychosocial, patient-centred and relationship-centred approaches to teaching about medical communication.     

Evaluating medical students' non-verbal communication during the objective structured clinical examination

OBJECTIVES: Non-verbal communication (NVC) in medical encounters is an important method of exchanging information on emotional status and contextualising the meaning of verbal communication. This study aimed to assess the impact of medical students' NVC on interview evaluations by standardised patients (SPs). METHODS: A total of 89 medical interviews in an objective structured clinical examination (OSCE) for post-clerkship medical students were analysed. All interviews were videotaped and evaluated on 10 non-verbal behaviour items. In addition, the quality of the interview content was rated by medical faculty on 5 items and the interview was rated by SPs on 5 items. The relationships between student NVC and SP evaluation were examined by multivariate regression analyses controlling for the quality of the interview content. RESULTS: Standardised patients were likely to give higher ratings when students faced them directly, used facilitative nodding when listening to their talk, looked at them equally when talking and listening, and spoke at a similar speed and voice volume to them. These effects of NVC remained significant after controlling for the quality of the interview content. CONCLUSIONS: This study provided evidence of specific non-verbal behaviours of doctors that may have additional impacts on the patient's perception of his or her visit, independently of the interview content. Education in basic NVC should be incorporated into medical education alongside verbal communication.     

Can communication skills be assessed independently of their context?

In many examinations, communication skills tend to be treated as if they are a single attribute independent of the context of the communication. However, it is clear that such assessments are confounded by candidates' knowledge or lack of knowledge of the medical issues about which they are communicating. In the 1990 Part One examination for Membership of the Royal New Zealand College of General Practitioners candidates were provided with all the essential knowledge relevant to the problem they were to communicate about. Despite this, performance was still seen to be context specific, demonstrating that such specificity is not purely knowledge related. Candidates completing the examination were observed to share information about the cases with candidates about to commerce. There was no evidence that performance was enhanced by such breaches in examination security.     

An analysis of standardised patient checklist errors and their effect on student scores

BACKGROUND: The frequency and nature of standardised patient (SP) recording errors during clinical performance examinations (CPX) have an effect on case scores and ultimately on pass/fail decisions. PURPOSE: To determine the effect of SP recording errors on case scores. METHODS: Standardised patients completed checklists immediately after each encounter. To determine checklist accuracy, multiple reviewers developed a checklist key for each student encounter studied. The total errors, the net errors, the errors of commission and omission and error rates by competency skill were analysed. RESULTS: The frequency of errors in history taking was greater than in physical examination, and the majority of errors were made in the students' favour. Summing the errors of commission and omission decreased the effect of total errors on student scores. CONCLUSIONS: High levels of SP recording accuracy are achievable. When errors occur, the net effect is usually in the students' favour.     

Developing a Korean communication skills attitude scale: comparing attitudes between Korea and the West

Objectives  We aimed to develop a Korean version of the Communication Skills Attitude Scale (CSAS) created by Rees et al . in order to elucidate the positive and negative aspects of Korean pre-medical and medical students' attitudes towards communication skills (CS) learning.
Methods  We performed two surveys. In the first of these, 325 pre-medical and medical students completed a translated version of the CSAS. In the second survey, 257 medical students and doctors-in-training answered five open-ended questions to obtain more qualitative data about their attitudes.
Results  Principal component analysis with direct oblimin rotation performed with the data from the first survey produced the following five factors: facilitation of interpersonal skills; doubts about the importance of CS learning in medicine; motivation; negative attitudes towards assessment, and overconfidence. Results from the second survey indicated that facilitation and importance within a medical context were two core attitudinal factors and suggested some modification to the CSAS to improve its fit for Korean pre-medical and medical students.
Conclusions  Using a Korean version of the CSAS (CSAS-K), we determined five factors that revealed a somewhat complex attitude structure among students towards CS learning. The CSAS required some modification, possibly because CS teaching and learning in Korea are in the development stage. Finally, the educational implications of the results are discussed.     

Follow the patient: process and outcome evaluation of medical students' educational experiences accompanying outpatients

BACKGROUND: To instill patient-centred attitudes in medical students, several medical schools in Japan have recently started to offer educational experiences in which medical students accompany outpatients throughout entire visits to hospitals. OBJECTIVE: To evaluate the processes and outcomes of the educational experience of Year 5 medical students accompanying outpatients at Nagoya University Hospital. METHODS: An integrated, multimethod approach was adopted using a written survey with open-ended questions for students, focus groups with students, and a written evaluation survey for patients. In all, 99 students completed the survey, 19 students participated in 3 focus groups, and 46 patients participated in the evaluation. RESULTS: Many students were sceptical about the objectives of the exercise. We were able to gain insight into student perceptions about facets of the exercise such as the ratio of students to patients and whether or not students should wear white coats. In particular, there was consensus among students about the importance of the debriefing session after the experience. Students achieved different learning outcomes depending on their particular individual experiences. In the student survey, 49% were satisfied with this experience, 6% were dissatisfied, and 43% were neither. In contrast, patients were highly satisfied with the experience (mean score 4.2 out of 5.0 on a Likert scale). Some students expressed concern about being a burden to patients, while many patients reported feeling emotionally supported by being accompanied by students. CONCLUSION: An integrated approach to programme evaluation, using quantitative and qualitative methods, was useful in the process and outcome evaluation of this new educational experience. The results have been taken into consideration for quality improvement of this curricular element.