多元概化理论在医学生毕业客观结构化临床考试中的应用

目的评估以标准化病人(SP)为基础的客观结构化临床考试(OSCE)在医学生临床技能考试的信度。方法用多元概化理论对临床医学生毕业考试OSCE中SP为基础的临床技能考核结果进行分析。结果本次OSCE总的相对概化系数为0.886,绝对概化系数为0.883,说明本次考核的信度较高。结论多元概化理论能客观、科学地评价毕业考试OSCE中SP考站的考试结果,其分析结果对于提高及改进临床技能考核质量有较大帮助。     

Can written communication skills be tested in an objective structured clinical examination format?

PURPOSE: To design and evaluate an objective structured clinical examination (OSCE) station on dictating a consult letter as part of a formative OSCE for internal medicine residents. METHOD: A 22-minute station for the dictation of a consult letter was included in a ten-station OSCE. Two raters completed a 34-item rating scale for 36 letters. The rating scale involved content and style. The station's score was derived from an overall rating of each section of the letter (history, physical examination, impression/plan) and a global rating of the complete letter. The exam also contained a physical exam station on the same patient problem and a verbal communication station. Residents provided written feedback following the station. RESULTS: The inter-rater reliability for the station's score was .72. The generalizability coefficient for the two-rater four-part rating scale was .79. The correlation between the consult letter and the total exam scores was .56, the highest for all ten stations. A significant correlation existed between the verbal communication station's score and the letter station's score (r =.37, p <.005), but no correlation occurred between the physical exam station's score on a similar patient problem and the letter station's score. The feedback from residents was favorable regarding amount of information provided and time allotted. CONCLUSION: An OSCE station is a feasible way to examine written communication skills. Letter-writing skills appear to be distinct from knowledge (physical exam station), but somewhat linked to verbal communication skills.     

Implementing the objective structured clinical examination in a traditional medical school

Since 1985 at the University of North Carolina at Chapel Hill School of Medicine, an objective structured clinical examination (OSCE) has been implemented at the completion of the second year as the final examination in physical diagnosis. This paper describes the format of the examination and the results of surveys of students and evaluators made in 1987, 1988, and 1989. Both the students and the evaluators (who were faculty, residents, and fellows) showed overwhelming acceptance and support of the OSCE. The data indicate that this reaction is partially because the examination is structured as a mechanism for teaching as well as for evaluation.     

The effect of candidates' perceptions of the evaluation method on reliability of checklist and global rating scores in an objective structured clinical examination.

PURPOSE: Process-oriented global ratings, which assess "overall performance" on one or a number of domains, have been purported to capture nuances of expert performance better than checklists. Pilot data indicate that students change behaviors depending on their perceptions of how they are being scored, while experts do not. This study examines the impact of the students' orientation to the rating system on OSCE scores and the interstation reliability of the checklist and global scores. METHOD: A total of 57 third- and fourth-year medical students at one school were randomly assigned to two groups and performed a ten-station OSCE. Group 1 was told that scores were based on checklists. Group 2 was informed that performance would be rated using global ratings geared toward assessing overall competence. All candidates were scored by physician-examiners who were unaware of the students' orientations to the rating system and who used both checklists and global rating forms. RESULTS: A mixed two-factor ANOVA identified a significant interaction of rating form by group (F(1,55) = 5.5, p <.05), with Group 1 (checklist-oriented) having higher checklist scores but lower global scores than did Group 2 (oriented to global ratings). In addition, Group 1 had higher interstation alpha coefficients than did Group 2 for both global scores (0.74 versus 0.63) and checklist scores (0.63 versus 0.40). CONCLUSIONS: The interaction effect on total exam scores suggests that students adapt their behaviors to the system of evaluation. However, the lower reliability coefficients for both forms found in the process-oriented global-rating group suggest that an individual's capacity to adapt to the system of global rating forms is relatively station-specific, possibly depending on his or her expertise in the domain represented in each station.     

临床医学专业“七转八”入学考试增加客观结构化临床考试实践探索

目的招收优秀的临床医学七年制医学生进入北京协和医学院临床医学八年制"七转八"项目学习。方法在临床医学专业"七转八"项目入学考试中增加5个考站的客观结构化临床考试(OSCE)。结果对考试成绩进行分析显示,OSCE的总成绩、笔试总分和沟通交流等项目近乎于正态分布,测试能力较好;SP问诊、查体和英语笔试没能区分出差异。考生的笔试成绩与各考站的相关系数在0.048~0.334之间。结论 2014年"七转八"入学OSCE项目的设计是成功的,有必要在"七转八"入学考试增加OSCE。     

Learning through examinations: use of an objective structured clinical examination as a teaching method in general practice

An objective structured clinical examination was written to evaluate the primary care teaching in Southampton medical school's third year curriculum. Its transition from a summative to a formative evaluation is described. During the academic year 1985-86 the examination was completed by 115 students in 12 groups, and the results give some support to its validity. Students reported that they had learnt from the exercise and the reasons for this are discussed.     

客观结构化临床考试在住院医师规范化培训中的分层管理作用初探

运用客观结构化临床考试(OSCE)评估新入院规范化培训住院医师,分析考核结果,并据此结果对新住院医师进行分层管理和分层教学,可提高规范化培训的效率和效果。     

Web-based objective structured clinical examination with remote standardized patients and Skype: Resident experience

Objective

Using Skype and remote standardized patients (RSPs), investigators sought to evaluate user acceptance of a web-based objective structured clinical examination (OSCE) among resident physicians.

Methods

After participating in four web-based clinical encounters addressing pain with RSPs, 59 residents from different training programs, disciplines and geographic locations completed a 52-item questionnaire regarding their experience with Skype and RSPs. Open-ended responses were solicited as well.

Results

The majority of participants (97%) agreed or strongly agreed the web-based format was convenient and a practical learning exercise, and 90% agreed or strongly agreed the format was effective in teaching communication skills. Although 93% agreed or strongly agreed they could communicate easily with RSPs using Skype, 80% preferred traditional face-to-face clinical encounters, and 58% reported technical difficulties during the encounters. Open-ended written responses supported survey results.

Conclusion

Findings from this study expose challenges with technology and human factors, but positive experiences support the continued investigation of web-based OSCEs as a synchronous e-learning initiative for teaching and assessing doctor–patient communication. Such educational programs are valuable but unlikely to replace face-to-face encounters with patients.

Practice implications

This web-based OSCE program provides physician learners with additional opportunity to improve doctor–patient communication.     

临床医学毕业考试之客观结构化临床考试的实施效果调查

探讨北京协和医学院毕业考试过程存在的问题,对考生进行问卷调查.问卷从考试的必要性、设计、组织及实施、标准化病人的表现、考核临床综合能力的效果、对今后学习的帮助、考试的公平性合理性客观性、及考试存在的问题和建议等方面进行了调查,客观结构化临床考试是一个相对公平、有效的临床技能综合评估方式,得到师生的认可,但在具体组织和实施方面需要更加关注专业性和细节性,建立客观结构化临床考试标准化流程.     

无纸化客观结构化临床考试(OSCE)应用探索

客观结构化临床考试在临床医学教育测评中应用日益广泛,但其复杂的组织流程也为考务工作者带来大量繁琐的考务工作,不利于考试的标准化实施。为应对这个问题,北京协和医学院设计了无纸化OSCE考试系统,该系统有无纸化考试的优点,也存在一些不足,最大的问题是对考试设备及网络太过依赖,如果考试设备故障或者网络系统瘫痪的情况下,无纸化考试将无法运行,考务工作者应还需要准备1~2套应对方案。     

Use of the objective structured clinical examination for assessment of vocational trainees for general practice

General practice training schemes currently have no structured methods of assessment and most rely on a variety of subjective ratings of performance. In West Cumbria the `objective structured clinical examination' has been used to assess training performance in areas covered by small group teaching during the preceding terms. Consultation skills, interpretation of clinical data and a number of aspects of practice management were tested. The examination was conducted in the local postgraduate centre and assessed 20 trainees. Each trainee received feedback of his performance on each problem set and also an overall comparison with his peers.

This method of assessment appeared to be well received by trainees and was practicable within the limited resources available. In addition, the variety of problems set allowed for a broad range of trainees' performances to be assessed.

     

Situational awareness within objective structured clinical examination stations in undergraduate medical training - a literature search

Background

Medical students may not be able to identify the essential elements of situational awareness (SA) necessary for clinical reasoning. Recent studies suggest that students have little insight into cognitive processing and SA in clinical scenarios. Objective Structured Clinical Examinations (OSCEs) could be used to assess certain elements of situational awareness. The purpose of this paper is to review the literature with a view to identifying whether levels of SA based on Endsley’s model can be assessed utilising OSCEs during undergraduate medical training.

Methods

A systematic search was performed pertaining to SA and OSCEs, to identify studies published between January 1975 (first paper describing an OSCE) and February 2017, in peer reviewed international journals published in English. PUBMED, EMBASE, PsycINFO Ovid and SCOPUS were searched for papers that described the assessment of SA using OSCEs among undergraduate medical students. Key search terms included “objective structured clinical examination”, “objective structured clinical assessment” or “OSCE” and “non-technical skills”, “sense-making”, “clinical reasoning”, “perception”, “comprehension”, “projection”, “situation awareness”, “situational awareness” and “situation assessment”. Boolean operators (AND, OR) were used as conjunctions to narrow the search strategy, resulting in the limitation of papers relevant to the research interest. Areas of interest were elements of SA that can be assessed by these examinations.

Results

The initial search of the literature retrieved 1127 publications. Upon removal of duplicates and papers relating to nursing, paramedical disciplines, pharmacy and veterinary education by title, abstract or full text, 11 articles were eligible for inclusion as related to the assessment of elements of SA in undergraduate medical students.

Discussion

Review of the literature suggests that whole-task OSCEs enable the evaluation of SA associated with clinical reasoning skills. If they address the levels of SA, these OSCEs can provide supportive feedback and strengthen educational measures associated with higher diagnostic accuracy and reasoning abilities.

Conclusion

Based on the findings, the early exposure of medical students to SA is recommended, utilising OSCEs to evaluate and facilitate SA in dynamic environments.

     

A new method for the assessment of patient safety competencies during a medical school clerkship using an objective structured clinical examination

INTRODUCTION:

Patient safety is seldom assessed using objective evaluations during undergraduate medical education.

OBJECTIVE:

To evaluate the performance of fifth-year medical students using an objective structured clinical examination focused on patient safety after implementation of an interactive program based on adverse events recognition and disclosure.

METHODS:

In 2007, a patient safety program was implemented in the internal medicine clerkship of our hospital. The program focused on human error theory, epidemiology of incidents, adverse events, and disclosure. Upon completion of the program, students completed an objective structured clinical examination with five stations and standardized patients. One station focused on patient safety issues, including medical error recognition/disclosure, the patient-physician relationship and humanism issues. A standardized checklist was completed by each standardized patient to assess the performance of each student. The student''s global performance at each station and performance in the domains of medical error, the patient-physician relationship and humanism were determined. The correlations between the student performances in these three domains were calculated.

RESULTS:

A total of 95 students participated in the objective structured clinical examination. The mean global score at the patient safety station was 87.59±1.24 points. Students'' performance in the medical error domain was significantly lower than their performance on patient-physician relationship and humanistic issues. Less than 60% of students (n = 54) offered the simulated patient an apology after a medical error occurred. A significant correlation was found between scores obtained in the medical error domains and scores related to both the patient-physician relationship and humanistic domains.

CONCLUSIONS:

An objective structured clinical examination is a useful tool to evaluate patient safety competencies during the medical student clerkship.     

An objective structured clinical examination for the licentiate: report of the pilot project of the Medical Council of Canada.

The Medical Council of Canada (MCC) administers a qualifying examination for the issuance of a license to practice medicine. To date, this examination does not test the clinical skills of history taking, physical examination, and communication. The MCC is implementing an objective structured clinical examination (OSCE) to test these skills in October 1992. A pilot examination was developed to test the feasibility, reliability, and validity of running a multisite, two-form, four-hour, 20-station OSCE for national licensure. In February 1991, 240 volunteer first- and second-year residents were tested at four sites. The candidates were randomly assigned to one of two forms of the test and one of two sites for two of the four sites. Generalizability analysis revealed that the variance due to form was 0.0 and that due to site was .16 compared with a total variance of 280.86. The reliabilities (inter-station) were .56 and .60 for the two forms. Station total-test score correlations, used to measure station validity, were significant for 38 of the 40 stations used (range .14-.60). The results of the OSCE correlated moderately with the MCC qualifying examination; these correlations were .32 and .35 for the two test forms. Content validity was assessed by postexamination questionnaires given to the physician examiners using a scale of 0 (low) to 10 (high). The physicians' mean ratings were: importance of the stations, 8.1 (SD, 1.8); success of the examination in testing core skills, 8.1 (SD, 1.6); and degree of challenge, 7.8 (SD, 2.1). The results indicate that a full-scale national administration of an OSCE for licensure is feasible using the model developed. Aspects of validity have been established and strategies to augment reliability have been developed.