Implementing the objective structured clinical examination in a geriatrics fellowship program-a 3-year experience

The Objective Structured Clinical Examination (OSCE) appears to be an effective alternative for assessing not only medical knowledge, but also clinical skills, including effective communication and physical examination skills. The purpose of the current study was to implement an OSCE model in a geriatrics fellowship program and to compare the instrument with traditional essay examination. Seventy first- and second-year geriatric fellows were initially submitted to a traditional essay examination and scored from 0 to 10 by a faculty member. The same fellows subsequently underwent an OSCE with eight 10-minute stations covering a wide range of essential aspects of geriatric knowledge. Each OSCE station had an examiner responsible for its evaluation according to a predefined checklist. Checklist items were classified for analysis purposes as clinical knowledge items (CKI) and communication skills items (CSI); fellow responses were scored from 0 to 10.Although essay examinations took from 30 to 45 minutes to complete, 180-200 minutes were required to evaluate fellows using the proposed OSCE method. Fellows scored an average of 6.2 ± 1.2 on the traditional essay examination and 6.6 ± 1.0 on the OSCE (P < .001). Subanalyses of OSCE scores indicated that average performance on CKI was lower than the average on CSI (6.4 ± 1.1 vs. 8.4 ± 1.1; P < .001). Fellow performance on the essay examination was similar to their performance on CKI (P = .13). Second-year fellows performed better than first-year fellows on the essay examination (P < .001) and CKI (P = .05), but not on CSI (P = .25).The OSCE was successfully implemented as an educational strategy during a geriatrics fellowship program. Combining different testing modalities may provide the best assessment of competence for various domains of knowledge, skills, and behavior.     

Effect of a mandatory geriatric medicine clerkship on third-year students

A nationwide push has increased geriatric medicine instruction within medical school curricula. Some institutions have proceeded with an integrated 4-year curriculum while others have constructed discrete courses in the third or fourth year of medical school. This paper describes the impact of a new mandatory 4-week geriatric medicine clerkship on third-year students developed by the Donald W. Reynolds Department of Geriatric Medicine at the University of Oklahoma Health Sciences Center. In the first year of implementation, 135 students took the course on both the Oklahoma City and Tulsa campuses. Clinical sites included inpatient, VA extended care unit, outpatient clinics, dementia clinics, home care, long-term care settings, and hospice. Didactic instruction used formal lectures and problem-based learning. The impact of the clerkship on students was assessed in three areas: knowledge, skills, and attitude using a pre- and postknowledge test, student satisfaction survey, and written comments. This article discusses how the clerkship resulted in increased knowledge of geriatric medicine. Student self-report indicates that the clerkship enhanced clinical evaluation and patient assessment skills. Students indicated that the experience was positive and recognized the importance of geriatric medicine in their development as doctors.     

The structure of medical competence and results of an OSCE

BACKGROUND: Medical competence is a central concept in medical education. Educational efforts in medical training are directed at the achievement of a maximal medical competence. The concept of the structure of medical competence (multidimensional or one-dimensional with strongly interrelated competences) therefore affects the educational developments and assessment procedures. PURPOSE: To examine the applicability of a one or more dimensional character of medical competence in student assessments, by analysing the results of 356 students in the history taking station of an objective structured clinical examination (OSCE), in relation to other assessment procedures. METHODS: The performances of 356 students in a history taking station of an OSCE were analysed. Analyses of the checklist scores were aimed at the dimensionality of history taking skills. External criteria were used to test the validity of the scores on the checklist. RESULTS: The analyses of the scores on the history taking checklist indicated at least five dimensions of history taking skills: the frequency of patient-centred skills, the quality of performance of patient-centred skills, complaint-oriented skills, general social skills, and the provision of procedural information. CONCLUSION: Medical competence, as a subject of assessment, can be seen as a multifaceted construct. This study shows that history taking alone might be composed of five different dimensions, suggesting that medical competence in respect of assessment might be viewed as a multifaceted construct which in that sense has implications for the assessment of medical competence.     

Development and implementation of an objective structured clinical examination to provide formative feedback on communication and interpersonal skills in geriatric training

Teaching and assessment of communication and interpersonal skills, one of the American Council for Graduate Medical Education-designated core competencies, is an important but difficult task in the training of physicians. Assessment of trainees offers an opportunity to provide explicit feedback on their skills and encourages learning. This article describes a pilot study in which clinician-educators affiliated with the geriatrics training programs at Beth Israel Deaconess Medical Center and Boston University Medical Center designed and piloted a novel Objective Structured Clinical Examination (OSCE) to assess the communication and interpersonal skills of medical, dental, and geriatric psychiatry fellows. The OSCE consisted of three stations where geriatricians and standardized patients evaluated candidates using specifically designed checklists and an abbreviated version of the Master Interview Rating Scale. Communication skills were assessed through performance of specific "real life" clinical tasks, such as obtaining a medical history, explaining a diagnosis and prognosis, giving therapeutic instructions, and counseling. Interpersonal skills were assessed through the effect of the communication between doctor and standardized patient on fostering trust, relieving anxiety, and establishing a therapeutic relationship. This pilot study demonstrated that the OSCE format of assessment provides a valid means of evaluating the communication and interpersonal skills of interdisciplinary geriatric trainees and provides a valuable forum for formative assessment and feedback. Given that geriatricians and non geriatricians involved in elder care both need communication and interpersonal skills, this novel OSCE can be used for assessment of these skills in trainees in diverse healthcare subspecialties.     

Relationship Between Peer Assessment During Medical School, Dean’s Letter Rankings, and Ratings by Internship Directors

BACKGROUND: It is not known to what extent the dean's letter (medical student performance evaluation [MSPE]) reflects peer-assessed work habits (WH) skills and/or interpersonal attributes (IA) of students. OBJECTIVE: To compare peer ratings of WH and IA of second- and third-year medical students with later MSPE rankings and ratings by internship program directors. DESIGN AND PARTICIPANTS: Participants were 281 medical students from the classes of 2004, 2005, and 2006 at a private medical school in the northeastern United States, who had participated in peer assessment exercises in the second and third years of medical school. For students from the class of 2004, we also compared peer assessment data against later evaluations obtained from internship program directors. RESULTS: Peer-assessed WH were predictive of later MSPE groups in both the second (F = 44.90, P < .001) and third years (F = 29.54, P < .001) of medical school. Interpersonal attributes were not related to MSPE rankings in either year. MSPE rankings for a majority of students were predictable from peer-assessed WH scores. Internship directors' ratings were significantly related to second- and third-year peer-assessed WH scores (r = .32 [P = .15] and r = .43 [P = .004]), respectively, but not to peer-assessed IA. CONCLUSIONS: Peer assessment of WH, as early as the second year of medical school, can predict later MSPE rankings and internship performance. Although peer-assessed IA can be measured reliably, they are unrelated to either outcome.     

Teaching about substance abuse with objective structured clinical exams

BACKGROUND: Although residents commonly manage substance abuse disorders, optimal approaches to teaching these specialized interviewing and intervention skills are unknown. OBJECTIVE: We developed a Substance Abuse Objective Structured Clinical Exam (OSCE) to teach addiction medicine competencies using immediate feedback. In this study we evaluated OSCE performance, examined associations between performance and self-assessed interest and competence in substance abuse, and assessed learning during the OSCE. DESIGN: Five-station OSCE, including different substance abuse disorders and readiness to change stages, administered during postgraduate year-3 ambulatory rotations for 2 years. PARTICIPANTS: One hundred and thirty-one internal and family medicine residents. MEASUREMENTS: Faculty and standardized patients (SPs) assessed residents' general communication, assessment, management, and global skills using 4-point scales. Residents completed a pre-OSCE survey of experience, interest and competence in substance abuse, and a post-OSCE survey evaluating its educational value. Learning during the OSCE was also assessed by measuring performance improvement from the first to the final OSCE station. RESULTS: Residents performed better (P<.001) in general communication (mean+/-SD across stations=3.12+/-0.35) than assessment (2.65+/-0.32) or management (2.58+/-0.44), and overall ratings were lowest in the contemplative alcohol abuse station (2.50+/-0.83). Performance was not associated with residents' self-assessed interest or competence. Perceived educational value of the OSCE was high, and feedback improved subsequent performance. CONCLUSIONS: Although internal and family medicine residents require additional training in specialized substance abuse skills, immediate feedback provided during an OSCE helped teach needed skills for assessing and managing substance abuse disorders.     

Emergency medicine management of the geriatric patient: an educational program for medical students

This article describes the experience of fourth-year medical students participating in a geriatric education program integrated into a 4-week emergency medicine student clerkship. Between July 2002 and April 2003, all students in this required clerkship participated in a geriatric educational program consisting of a small group discussion of medical and psychosocial issues of older adult emergency department (ED) patients. Students used learned skills to evaluate older adult ED patients for medical and psychosocial issues and later followed up with these patients by telephoning them at their homes or visiting them in the hospital. Students tracked their evaluations of the medical problems, functional abilities, and social supports of patients in the ED. Students also noted when their assessments resulted in the acquisition of new skills or knowledge and when their evaluation of geriatric syndromes resulted in a change of the patient care plan. Seventy-seven students evaluated 217 patients in the ED, of whom 167 (77%) received a follow-up visit or phone call. Students documented learning new skills while caring for 80 (48%) of the older adult patients. Qualitative survey responses from students indicated that students had increased understanding of the importance of assessing functional status and social supports and providing interdisciplinary care. Integrating geriatric education modules into existing emergency medicine clerkships is an effective method to expand the geriatric curriculum in medical schools and to emphasize the importance of geriatric assessment and syndromes in emergency care.     

Cluster randomized controlled trial of the impact of a computer-assisted learning package on the learning of musculoskeletal examination skills by undergraduate medical students

OBJECTIVE: To identify whether there was measurable impact of a specific computer-assisted learning (CAL) package, "Virtual Rheumatology," on the learning of musculoskeletal examination skills by medical students. METHODS: We conducted 2 parallel, cluster-randomized controlled trials using undergraduate curricula at 2 locations: Newcastle and London, UK. Medical students attending a musculoskeletal rotation were allocated to the intervention (Virtual Rheumatology CD) or the control arm of the study by placement group. A formative 14-item objective structured clinical examination (OSCE) assessment on the examination of shoulder and/or knee joints was the main outcome measure at Newcastle. At London, a 17-item knee station formed part of the summative OSCE. We also used a questionnaire including a 15-item confidence log (C-Log) for self assessment of musculoskeletal examination skills and knowledge. Analysis was by intention to teach. RESULTS: At Newcastle, there were 112 students in the CD allocated group and 129 in the non-CD group. The CD allocated group performed significantly better on the OSCE (P = 0.002) and C-Log (P = 0.005) than the non-CD group. At London, there were 48 students in the CD allocated group and 65 in the non-CD group. The CD allocated group performed better on the knee OSCE than the non-CD group (adjusted P = 0.040), but there was little difference in the change in C-Log scores from baseline to followup between the 2 groups (P = 0.582). CONCLUSION: The Virtual Rheumatology CD has a positive impact on the acquisition of musculoskeletal examination skills in medical students. Further study is needed to see if similar advantages could be gained in other clinical specialities and how CAL resources could be effectively integrated into the medical curriculum.     

Assessing core clinical competencies required of medical graduates in Taiwan

Medical students are assumed to be competent to provide basic patient care independently on graduation. However, there is a gap between what students are expected to learn and what they have actually learned. This may be due to the lack of clearly defined learning objectives, well-organized curriculum, and properly administered assessment. In an attempt to tackle this problem, we conducted a three-step study. Firstly, we identified the core clinical competencies required of medical graduates in Taiwan. Secondly, we incorporated these clinical competencies into a new medical curriculum. Finally, we identified the most appropriate assessment methods for each clinical competency. In 2004, a set of minimally required clinical competencies for medical undergraduates in Taiwan was developed, which included 92 clinical skills, four communication skills, and seven kinds of attitudes. In order to prepare 3rd and 4th year medical students at Kaohsiung Medical University (KMU) for later clinical work, the medical curriculum committee integrated the teaching and assessment of the core clinical skills identified previously into relevant organ-system blocks of the new curriculum. To identify appropriate assessment methods for each clinical skill, a structured questionnaire of assessment methods based on the Toolbox of Assessment Methods (Accreditation Council for Graduate Medical Education) and The Scottish Doctor (Scottish Deans' Medical Curriculum Group) was developed and distributed to 40 senior clinical faculty members at KMU. Simulations and Models, Standardized Patient Examination (SP), and Objective Structured Clinical Examination (OSCE) were suggested to be most suitable to assess two-thirds of the core clinical skills. These assessment methods are commonly used in American and European medical schools. We believe that the implementation of the new curriculum at KMU accompanied by the use of Simulations and Models, SP, OSCE, and other teaching and assessment methods will help 3rd and 4th year students to prepare better for clinical practice in clerkships.     

Improving the physical diagnosis skills of third-year medical students

OBJECTIVE: To determine if a literature-based physical diagnosis curriculum could improve student knowledge, skill, and self-confidence in physical diagnosis. DESIGN: Prospective controlled trial of an educational intervention. SETTING: Required internal medicine clerkship for third-year medical students at Brown Medical School. PARTICIPANTS: Third-year medical students who completed the internal medicine clerkship during the academic year 1999-2000: 32 students at 1 clerkship site received the intervention; a total of 50 students at 3 other clerkship sites served as controls. INTERVENTION: Physical diagnosis curriculum based on 8 articles from the Journal of the American Medical Association's Rational Clinical Examination series. Intervention students met weekly for 1 hour with a preceptor to review each article, discuss the sensitivity and specificity of the maneuvers and findings, and practice the techniques with an inpatient who agreed to be visited and examined. MEASUREMENTS AND MAIN RESULTS: Physical diagnosis knowledge for the 8 topics was evaluated using a 22-item multiple choice question quiz, skill was evaluated using trained evaluators, and self-confidence was assessed using an end-of-clerkship survey. Intervention students scored significantly higher than the control group on the knowledge quiz (mean correct score 70% vs 63%, P =.002), skills assessment (mean correct score 90% vs 54%, P <.001), and self-confidence score (mean total score 40 vs 35, P =.003), and they expressed greater satisfaction with the physical diagnosis teaching they received in the clerkship. CONCLUSION: This physical diagnosis curriculum was successful in improving students' knowledge, skill, and self-confidence in physical diagnosis.     

Using a multimedia tool to improve cardiac auscultation knowledge and skills

OBJECTIVE: Today's medical school graduates have significant deficits in physical examination skills. Medical educators have been searching for methods to effectively teach and maintain these skills in students. The objective of this study was to determine if an auscultation curriculum centered on a portable multimedia CD-ROM was effective in producing and maintaining significant gains in cardiac auscultatory skills. DESIGN: Controlled cohort study. PARTICIPANTS: All 168 third-year medical students at 1 medical school in an academic medical center. INTERVENTIONS: Students were tested before and after exposure to 1 or more elements of the auscultation curriculum: teaching on ward/clinic rotations, CD-ROM comprehensive cases with follow-up seminars, and a CD-ROM 20-case miniseries. The primary outcome measures were student performance on a 10-item test of auscultation skill (listening and identifying heart sound characteristics) and a 30-item test of auscultation knowledge (factual questions about auscultation). A subset of students was tested for attenuation effects 9 or 12 months after the intervention. RESULTS: Compared with the control group (1 month clinical rotation alone), students who were also exposed to the CD-ROM 20-case miniseries had significant improvements in auscultation skills scores (P < .05), but not knowledge. Additional months of clerkship, comprehensive CD-ROM cases, and follow-up seminars increased auscultation knowledge beyond the miniseries alone (P < .05), but did not further improve auscultation skills. Students' auscultation knowledge diminished one year after the intervention, but auscultation skills did not. CONCLUSION: In addition to the standard curriculum of ward and conference teaching, portable multimedia tools may help improve quality of physical examination skills.     

Effective teaching methods for geriatric competencies

This study assesses how effective classroom sessions are at teaching geriatric competencies to medical students. At Stony Brook Medical School, most geriatric competencies are taught in the Ambulatory Care Clerkship during small-group educational sessions. Clinical exposure to reinforce these specialized skills varies with preceptor assignment. A student's ability to perform geriatric assessments was evaluated by scores on an Objective Structured Clinical Exam (OSCE) with a geriatric patient. Scores from students who received additional clinical practice of these skills were compared with scores from students who did not. No significant difference in OSCE scores were seen between the two groups.     

Effective Teaching Methods for Geriatric Competencies

This study assesses how effective classroom sessions are at teaching geriatric competencies to medical students. At Stony Brook Medical School, most geriatric competencies are taught in the Ambulatory Care Clerkship during small-group educational sessions. Clinical exposure to reinforce these specialized skills varies with preceptor assignment. A student's ability to perform geriatric assessments was evaluated by scores on an Objective Structured Clinical Exam (OSCE) with a geriatric patient. Scores from students who received additional clinical practice of these skills were compared with scores from students who did not. No significant difference in OSCE scores were seen between the two groups.     

Cluster randomized controlled trial of the impact of a computer‐assisted learning package on the learning of musculoskeletal examination skills by undergraduate medical students

Objective

To identify whether there was measurable impact of a specific computer‐assisted learning (CAL) package, “Virtual Rheumatology,” on the learning of musculoskeletal examination skills by medical students.

Methods

We conducted 2 parallel, cluster‐randomized controlled trials using undergraduate curricula at 2 locations: Newcastle and London, UK. Medical students attending a musculoskeletal rotation were allocated to the intervention (Virtual Rheumatology CD) or the control arm of the study by placement group. A formative 14‐item objective structured clinical examination (OSCE) assessment on the examination of shoulder and/or knee joints was the main outcome measure at Newcastle. At London, a 17‐item knee station formed part of the summative OSCE. We also used a questionnaire including a 15‐item confidence log (C‐Log) for self assessment of musculoskeletal examination skills and knowledge. Analysis was by intention to teach.

Results

At Newcastle, there were 112 students in the CD allocated group and 129 in the non‐CD group. The CD allocated group performed significantly better on the OSCE (P = 0.002) and C‐Log (P = 0.005) than the non‐CD group. At London, there were 48 students in the CD allocated group and 65 in the non‐CD group. The CD allocated group performed better on the knee OSCE than the non‐CD group (adjusted P = 0.040), but there was little difference in the change in C‐Log scores from baseline to followup between the 2 groups (P = 0.582).

Conclusion

The Virtual Rheumatology CD has a positive impact on the acquisition of musculoskeletal examination skills in medical students. Further study is needed to see if similar advantages could be gained in other clinical specialities and how CAL resources could be effectively integrated into the medical curriculum.

     

A problem-based learning curriculum in geriatrics for medical students

ABSTRACT

A geriatrics curriculum delivered to medical students was evaluated in this study. Students were instructed to review real patient cases, interview patients and caregivers, identify community resources to address problems, and present a final care plan. Authors evaluated the course feedback and final care plans submitted by students for evidence of learning in geriatric competencies. Students rated the efficacy of the course on a 5-point Likert scale as 3.70 for developing clinical reasoning skills and 3.69 for interdisciplinary teamwork skills. Assessment of an older adult with medical illness was rated as 3.87 and ability to perform mobility and functional assessment as 3.85. Reviews of written final care plans provided evidence of student learning across several different geriatric competencies such as falls, medication management, cognitive and behavior disorders, and self-care capacity. Assessment of the curriculum demonstrated that medical students achieved in-depth learning across multiple geriatric competencies through contact with real cases.     

Implementation of an OSCE at Kaohsiung Medical University

The Objective Structured Clinical Examination (OSCE), a tool to objectively and fairly assess medical students' clinical competences, has become widely used in medical education worldwide. However, most medical schools in Taiwan have just begun to adopt this assessment method. In 2003, Kaohsiung Medical University (KMU) established the first standardized patient (SP) program in Taiwan and applied SPs with an OSCE. This study reports the process of the implementation of an OSCE at KMU, which includes collecting information, visiting leading clinical skills centers, consulting medical educators from other countries, holding international conferences, establishing an OSCE committee, writing cases, training SPs, administrating the OSCE, and receiving feedback from medical students. Most students were satisfied with the assessment and appreciated the learning experience. Based on the experience in 2003, the OSCE committee decided to incorporate the OSCE into the medical curriculum as a measure to assess medical students' clinical competences. In addition to assessing medical students' clinical competence, the OSCE can also be applied to other professional health education, such as dentistry, nursing, and pharmacy. We are currently sharing our experience with other colleges at KMU.     

Medical Students' Recognition and Application of Geriatrics Principles in a New Curriculum

Given the aging U.S. population, it is imperative that medical students recognize and apply geriatrics principles. To address this need, in 2006, the Warren Alpert Medical School of Brown University integrated geriatrics content into a new medical school curriculum. Preclinical and clinical medical students submitted written reflective journals in response to prompts regarding the geriatrics content of the new medical school curriculum, including their didactic and clinical experiences. An interdisciplinary team used a structured qualitative approach to identify themes, including the recognition and application of geriatrics principles. Thirty medical student journalers submitted 405 journal entries. Themes regarding students' emerging understanding of geriatrics principles included a growing understanding of geriatrics principles, recognition of the importance of psychosocial factors and patient preferences in caring for older adults, recognition of the complexities of treating older adults and application of geriatric principles to clinical situations, and understanding of physicians' roles in managing the care of older adults. Medical student reflective journaling allows medical educators to obtain timely feedback on curricular innovations and helps illuminate the process by which medical students learn to recognize and apply core geriatrics principles.     

Evaluation of patient partners in the teaching of the musculoskeletal examination

OBJECTIVE: To evaluate student preferences and examination outcomes of 2 different methods of teaching musculoskeletal (MSK) medicine examination techniques. METHODS: Year 2 students in a 4 year graduate medical school were randomized to 3 teaching groups: students in Group 1 were taught by rheumatology fellows, Group 2 by patient partners, while group 3 were randomly allocated to either patient partner (3A) or rheumatology trainee (3B) teaching. All students were debriefed at the end of each teaching block of 4 weeks for feedback on their teaching experience using a standardized questionnaire. In addition, at the end of the academic year, all students sat an objective structured clinical examination (OSCE) in clinical skills that contained a rheumatology station. The effect of method of teaching on students' performance in the rheumatology station was analyzed. RESULTS: Eighty medical students participated in the study. Overall, there was no difference in student ratings of either mode of teaching, although the students reported that patient partner teaching gave more opportunity to practice MSK examination skills and also provided greater feedback to the student. Students reported a preference for rheumatology trainee teaching because they believed the teaching would be more relevant to the content of the examination. There was no statistically significant difference in the performance of the students in the OSCE rheumatology station in regard to the mode of teaching they had received prior to the examination. CONCLUSION: Patient partner teaching is as effective a method of teaching clinical skills in MSK medicine as a traditional resident based form of teaching, with student benefits from patient feedback and greater "hands-on" opportunities. The assessment process should incorporate patient partners to assess the unique aspects of patient educator based teaching of MSK examination techniques.