Enhancing OSCE preparedness with video exemplars in undergraduate nursing students. A mixed method study

BackgroundObjective structured clinical examinations (OSCEs) are designed to assess clinical skill performance and competency of students in preparation for ‘real world’ clinical responsibilities. OSCEs are commonly used in health professional education and are typically associated with high levels of student anxiety, which may present a significant barrier to performance. Students, including nursing students, have identified that flexible access to exemplar OSCEs might reduce their anxiety and enable them to better prepare for such examinations.AimTo implement and evaluate an innovative approach to preparing students for OSCEs in an undergraduate (registration) acute care nursing course.MethodA set of digitized OSCE exemplars were prepared and embedded in the University-based course website as part of usual course learning activities. Use of the exemplars was monitored, pre and post OSCE surveys were conducted, and qualitative data were collected to evaluate the approach. OSCE grades were also examined.FindingsThe online OSCE exemplars increased self-rated student confidence, knowledge, and capacity to prepare and provided clarity around assessment expectations. OSCE exemplars were accessed frequently and positively received; but did not impact on performance.ConclusionVideo exemplars aid student preparation for OSCEs, providing a flexible, innovative and clear example of the assessment process. Video exemplars improved self-rated student confidence and understanding of performance expectations, leading to increased engagement and reduced anxiety when preparing for the OSCE, but not overall OSCE performance. Such OSCE exemplars could be used to increase staff capacity and improve the quality of the student learning experience.     

Task Demands in OSCEs Influence Learning Strategies

Theory: Models on pre-assessment learning effects confirmed that task demands stand out among the factors assessors can modify in an assessment to influence learning. However, little is known about which tasks in objective structured clinical examinations (OSCEs) improve students' cognitive and metacognitive processes. Research is needed to support OSCE designs that benefit students' metacognitive strategies when they are studying, reinforcing a hypothesis-driven approach. With that intent, hypothesis-driven physical examination (HDPE) assessments ask students to elicit and interpret findings of the physical exam to reach a diagnosis (“Examine this patient with a painful shoulder to reach a diagnosis”). Hypotheses: When studying for HDPE, students will dedicate more time to hypothesis-driven discussions and practice than when studying for a part-task OSCE (“Perform the shoulder exam”). It is expected that the whole-task nature of HDPE will lead to a hypothesis-oriented use of the learning resources, a frequent use of adjustment strategies, and persistence with learning. Method: In a mixed-methods study, 40 medical students were randomly paired and filmed while studying together for two hypothetical OSCE stations. Each 25-min study period began with video cues asking to study for either a part-task OSCE or an HDPE. In a crossover design, sequences were randomized for OSCEs and contents (shoulder or spine). Time-on-task for discussions or practice were categorized as “hypothesis-driven” or “sequence of signs and maneuvers.” Content analysis of focus group interviews summarized students' perception of learning resources, adjustment strategies, and persistence with learning. Results: When studying for HDPE, students allocate significantly more time for hypothesis-driven discussions and practice. Students use resources contrasting diagnoses and report persistence with learning. When studying for part-task OSCEs, time-on-task is reversed, spent on rehearsing a sequence of signs and maneuvers. Conclusions: OSCEs with similar contents but different task demands lead to opposite learning strategies regarding how students manage their study time. Measuring pre-assessment effects from a metacognitive perspective provides empirical evidence to redesign assessments for learning.     

Qualitative analysis of student beliefs and attitudes after an objective structured clinical evaluation: implications for affective domain learning in undergraduate nursing education

This qualitative study explored the feelings, beliefs, and attitudes of senior-level undergraduate pediatric nursing students upon completion of a medication administration Objective Structured Clinical Evaluation (OSCE). The affective domain is the most neglected domain in higher education, although it is deemed the "gateway to learning." Quantitative assessments of clinical skills performed during OSCEs usually address two of the three domains of learning: cognitive (knowledge) and psychomotor skills. Twenty students volunteered to participate in focus groups (10 per group) and were asked three questions relevant to their feelings, beliefs, and attitudes about their OSCE experiences. Students integrated the attitude of safety first into future practice but felt that anxiety, loss of control, reaction under pressure, and no feedback affected their ability to connect the OSCE performance with future clinical practice. The findings affect future affective domain considerations in the development, modification, and assessment of OSCEs across the undergraduate nursing curriculum.     

The relationship between second-year medical students' OSCE scores and USMLE Step 2 scores

RATIONALE, AIMS AND OBJECTIVES: A valid tool to measure clinical competency early in medical school could identify students who may require special educational attention. The overall aim is to assess the relationship between students' scores on an objective structured clinical examination (OSCE) given in the second year of medical school and their subsequent performance on Step 2 of the United States Medical Licensing Examination (USMLE Step 2). METHODS: Participants were 390 second-year medical students participating in a required OSCE; complete data (Medical College Admission Test, OSCE, USMLE Step 1 and Step 2 scores) were available for 340 students (87%). Univariate correlations and linear regression analyses were performed. RESULTS: Total OSCE score was moderately correlated with USMLE Step 2 score (r = 0.395, P < 0.001), as were two skills subscores of the OSCE, differential diagnosis (r = 0.343, P < 0.001) and identification of abnormality (r = 0.322, P < 0.001). In linear regression analysis, neither OSCE total score nor any of the subscores independently predicted Step 2 scores; only Step 1 score (beta = 0.687, P < 0.001) and female sex (beta = 0.152, P < 0.001) remained independent correlates of Step 2 score. CONCLUSION: OSCEs early in medical school can be useful in the early assessment of clinical competence.     

Students' perceptions of practice assessment in the skills laboratory: An evaluation study of OSCAs with immediate feedback

Assessment of clinical skills is fundamental to undergraduate nursing programmes. However, enabling assessment to be a good learning experience as well is a challenge to nurse educators. The study presented here presents the change from using an objective structured clinical examination (OSCE) for summative assessment (with feedback given to students after results had gone to the examination board – 6 weeks after the OSCE) to one with immediate feedback. Because the previous OSCEs were universally disliked by students, for reasons that included absence of immediate feedback, in making this change the university re-branded the OSCE as an objective structured clinical assessment (OSCA) with immediate feedback provided to students. A survey was undertaken to measure student engagement with the OSCA, its value and impact, and its sustainability from the students’ perspectives. There is little in the literature about student engagement with OSCEs and sustainability. Findings show that the OSCA with immediate feedback was perceived positively by students, was valued with regard to a number of factors, had a positively impact on student learning and confidence and was felt to be a form of assessment that this university should continue to use.     

The objective structured clinical examination within intensive care nursing education

Background: The objective structured clinical examination is commonly referred to as an ‘OSCE’ and is a way of assessing a student's competency with clinical skills. The OSCE involves the student demonstrating a skill during a simulated clinical situation in a controlled environment instead of using real patients in the practice setting. Post‐registered nursing courses are now beginning to use the OSCE as a form of summative assessment either as replacement for or in addition to skills assessed by a mentor in practice. Aim: This paper aims to reflect back on a recent experience of introducing an OSCE into a post‐registered, degree level intensive care nursing programme. Methods: The process of reflection will be used throughout the analysis following the Six Thinking Hats tool. Conclusions: Organizing and implementing an OSCE for an intensive care nursing programme required a great deal of preparation and time of the teacher but also offered various benefits and advantages compared with other forms of assessment. Disadvantages and limitations of the OSCE are also identified in this article along with recommendations and practical advice for other teachers, examiners and students participating in OSCEs.     

Implementing objective structured clinical skills evaluation (OSCE) in nurse registration programmes in a centre in Ireland: a utilisation focused evaluation

This paper presents findings from a research study of two objective structured clinical skills evaluation (OSCE) processes that were used in the new BSc. in General and Psychiatric Nursing Registration programmes offered in an Institute of Technology in the south west of Ireland. OSCEs, which have gained widespread acceptance throughout the western world as valid academic measures of nursing competence has yet to be established in this country. This research aimed to evaluate the process and outcomes of OSCE from the perspective of the major stakeholder groups: first and second year nursing students, lecturers, clinical placement coordinators and assessors. A utilisation focussed evaluation approach was adopted which values the opinions and experiences of stakeholders. It is intended to use the findings to develop the validity and quality of the OSCE assessment. The OSCE process was found to have a positive impact on all stakeholders. OSCEs were perceived to be a meaningful and fair form of assessment. Students identified that they felt more prepared for and more confident about forthcoming placements. The OSCE process was, however, perceived to be a stressful experience and requires considerable preparation effort by students and academic staff. Mature students claimed that more practice effort was required but also felt more prepared for placements and achieved higher OSCE scores.     

Standard Setting Methods for Pass/Fail Decisions on High-Stakes Objective Structured Clinical Examinations: A Validity Study

Construct: Authentic standard setting methods will demonstrate high convergent validity evidence of their outcomes, that is, cutoff scores and pass/fail decisions, with most other methods when compared with each other. Background: The objective structured clinical examination (OSCE) was established for valid, reliable, and objective assessment of clinical skills in health professions education. Various standard setting methods have been proposed to identify objective, reliable, and valid cutoff scores on OSCEs. These methods may identify different cutoff scores for the same examinations. Identification of valid and reliable cutoff scores for OSCEs remains an important issue and a challenge. Approach: Thirty OSCE stations administered at least twice in the years 2010–2012 to 393 medical students in Years 2 and 3 at Aga Khan University are included. Psychometric properties of the scores are determined. Cutoff scores and pass/fail decisions of Wijnen, Cohen, Mean–1.5SD, Mean–1SD, Angoff, borderline group and borderline regression (BL-R) methods are compared with each other and with three variants of cluster analysis using repeated measures analysis of variance and Cohen's kappa. Results: The mean psychometric indices on the 30 OSCE stations are reliability coefficient = 0.76 (SD = 0.12); standard error of measurement = 5.66 (SD = 1.38); coefficient of determination = 0.47 (SD = 0.19), and intergrade discrimination = 7.19 (SD = 1.89). BL-R and Wijnen methods show the highest convergent validity evidence among other methods on the defined criteria. Angoff and Mean–1.5SD demonstrated least convergent validity evidence. The three cluster variants showed substantial convergent validity with borderline methods. Conclusions: Although there was a high level of convergent validity of Wijnen method, it lacks the theoretical strength to be used for competency-based assessments. The BL-R method is found to show the highest convergent validity evidences for OSCEs with other standard setting methods used in the present study. We also found that cluster analysis using mean method can be used for quality assurance of borderline methods. These findings should be further confirmed by studies in other settings.     

Validity Evidence for Medical School OSCEs: Associations With USMLE® Step Assessments

Background: Recently, there has been a surge in the use of objective structured clinical examinations (OSCEs) at medical schools around the world, and with this growth has come the concomitant need to validate such assessments. Purposes: The current study examined the associations between student performance on several school-level clinical skills and knowledge assessments, including two OSCEs, the National Board of Medical Examiners® (NBME) Subject Examinations, and the United States Medical Licensing Examination® (USMLE) Step 2 Clinical Skills (CS) and Step 3 assessments. Methods: The sample consisted of 806 medical students from the Uniformed Services University of the Health Sciences. We conducted Pearson correlation analysis as well as stepwise multiple linear regression modeling to examine the strength of associations between students’ performance on 2nd- and 3rd-year OSCEs and their two Step 2 CS component scores and Step 3 scores. Results: Positive associations were found between the OSCE variables and the USMLE scores; in particular, student performance on both the 2nd- and 3rd-year OSCEs was more strongly associated with the two Step 2 CS component scores than with Step 3 scores. Conclusions: These findings, although preliminary, provide some predictive validity evidence for the use of OSCEs in determining readiness of medical students for clinical practice and licensure.     

The Associations Between Clerkship Objective Structured Clinical Examination (OSCE) Grades and Subsequent Performance

Construct: We investigated the extent of the associations between medical students' clinical competency measured by performance in Objective Structured Clinical Examinations (OSCE) during Obstetrics/Gynecology and Family Medicine clerkships and later performance in both undergraduate and graduate medical education. Background: There is a relative dearth of studies on the correlations between undergraduate OSCE scores and future exam performance within either undergraduate or graduate medical education and almost none on linking these simulated encounters to eventual patient care. Of the research studies that do correlate clerkship OSCE scores with future performance, these often have a small sample size and/or include only 1 clerkship. Approach: Students in USU graduating classes of 2007 through 2011 participated in the study. We investigated correlations between clerkship OSCE grades with United States Medical Licensing Examination Step 2 Clinical Knowledge, Clinical Skills, and Step 3 Exams scores as well as Postgraduate Year 1 program director's evaluation scores on Medical Expertise and Professionalism. We also conducted contingency table analysis to examine the associations between poor performance on clerkship OSCEs with failing Step 3 and receiving poor program director ratings. Results: The correlation coefficients were weak between the clerkship OSCE grades and the outcomes. The strongest correlations existed between the clerkship OSCE grades and the Step 2 CS Integrated Clinical Encounter component score, Step 2 Clinical Skills, and Step 3 scores. Contingency table associations between poor performances on both clerkships OSCEs and poor Postgraduate Year 1 Program Director ratings were significant. Conclusions: The results of this study provide additional but limited validity evidence for the use of OSCEs during clinical clerkships given their associations with subsequent performance measures.     

Partners for NP education: the 1999 AANP preceptor and faculty survey

PURPOSE: To increase understanding of national trends in nurse practitioner (NP) clinical education, the Education Committee of the American Academy of Nurse Practitioners (AANP) conducted a survey at the 1999 National AANP Conference in Atlanta. DATA SOURCES: A convenience sample of preceptors (n = 87) and faculty (n = 42) out of the total attendance of 1,744 responded to a written questionnaire. CONCLUSIONS: Preceptor respondents provided data concerning the number of NP students supervised, influence of student supervision on productivity, and availability of incentives for precepting. Faculty reported placement and supervision issues, the extent of precepting in their clinical practice site, and recognition and support for this role. Faculty and preceptors disagreed about the types and number of incentives offered for accepting students as well as the congruence of clinical teaching activities and national teaching guidelines. Precepting did not appear to strongly influence preceptor productivity. External funding did not influence opportunities for clinical education. IMPLICATIONS FOR PRACTICE: In 1998, the graduates of NP programs rose by 15.8% and over half of all nursing students enrolled in graduate nursing programs were seeking a NP education. This increase in students may compromise the ability of schools of nursing to insure quality clinical education of NP students by increasing faculty workload and placing greater demands on expert preceptors in the community. Clinical education is also changing in light of changes in the health care system.     

Administration of a parallel,simultaneous objective structured clinical examination to accommodate a large class of students

The use of an objective structured clinical examination (OSCE) to evaluate the clinical abilities of second‐year medical students at the end of an introduction to clinical medicine course in 2 successive years is reported. Due to the large number of students in our classes, two identical, simultaneous, parallel OSCEs were administered each year. Skills to be evaluated and cases used to measure these skills were determined by a modification of existing methods. The logistic feasibility of administering a large OSCE in this manner was confirmed. A thorough psychometric evaluation of the OSCE was performed, and findings were evaluated. When used in a pass‐fail context and calculated as a dependability index with cutoffs, the generalizability of the total OSCE and most individual skills measured was greater than .8 when the cutoff was 2 SD below the mean score. The number of cases required to achieve a generalizability of .8 for the total OSCE and each individual skill was fewer than 11. The potential for use of the OSCE in making pass‐fail decisions in medical school classes and proposals for modification of the mechanisms of administration and scoring of the OSCE are discussed.     

Correlation Between Student Performances on Course Level Integrated Clinical Skills Examinations and Objective Structured Clinical Examinations in a Chiropractic College Program

Purpose: This retrospective study measured correlation of student performance between 2 objective structured clinical examinations (OSCEs) and an introductory integrated clinical skills course that preceded the OSCEs. The hypothesis was that there would be a strong, positive correlation between the earlier level examinations and the upper level OSCE, high enough that earlier examinations could be viewed as predictors of upper level OSCE performance. Methods: Using student scores for 5 academic terms of upper level OSCEs for 2008–2009 (n = 208) and respective earlier scores, correlation coefficients were calculated for the upper level OSCE and Clinical Skills course, and upper and lower level OSCEs. Multiple linear regression analysis was used to evaluate how well the lower level OSCE and clinical skills scores, both as lone and combined independent variables, predicted the upper level OSCE scores. Results: There was at least a moderate correlation between both sets of scores: r = .51 (p < .001) between upper level OSCE and clinical skills course, r = .54 (p < .001) between the upper and lower level OSCEs. A combination of clinical skills and lower level OSCE scores suggested a moderate prediction of upper level OSCE scores (R² = .38.) Conclusions: Correlations were found to be of at least a moderate level. According to linear regression analysis, a combination of the earlier scores was moderately predictive for the upper level OSCE. More research could be done to determine additional components of student performance.Key Indexing Terms: Chiropractic, Education, Educational Measurement     

OSCEs--seven years on the bandwagon: the progress of an objective structured clinical evaluation programme

The original purpose of Objective Structured Clinical Examination (OSCE), as first described in the medical literature [Harden, R.M., Stevenson, M., Downie, W.W., Wilson, G.M., 1975. Assessment of clinical competence using objective structured examination. British Medical Journal, 1, 447-451], provided a means of examining the skills acquisition of medical students. A review of the literature, since that time, provides the background to the development of OSCEs into pre-registration nursing curricula, with the OSCE programme at the University of Salford presented here as a case study. The original student sample was a mixture of 150-250 adult, child and mental health students in each of seven cohorts over a period of four years. Each student undertook a 30-min formative, simulated patient, holistic care OSCE in their second year of the programme. Later developments included one remote workstation connected to and as part of the holistic patient care encounter. In subsequent curricula, the larger cohorts of around 250-300 students were accommodated in a formative rotational three-workstation OSCE, based on clinical skills to be acquired prior to their first clinical placement. A summative patient-centred OSCE was undertaken in practice at a later date. The educational and practice value of OSCEs regarding their clinical content and context in nursing curricula now and in the future are explored, along with the practicalities of implementation.     

Using video to support veterinary nursing students preparing for OSCEs

Objective structured clinical examinations (OSCEs) inspire fear and loathing in student veterinary nurses and clinical coaches alike. With different courses using different assessment criteria, students can find preparing for OSCEs stressful if they receive different advice from difference sources. Unfamiliarity with equipment and resources present in the examination can be a further source of anxiety; if a student uses one type of centrifuge in practice, but faces a different model in their OSCE this can trigger panic and shaking hands may struggle to fill capillary tubes, especially with the added pressure of an examiner armed with a clipboard standing in attendance!     

The cost of the objective structured clinical examination on an Italian nursing bachelor's degree course

The OSCE (Objective Structured Clinical Examination) is considered the most valid and reliable method for assessing the clinical skills of students training for health professions, but its use is limited by the related high costs. We analyzed the cost retrospectively of using an OSCE designed for second-year students (2009) in our degree course, adopting the Reznick et al. guidelines (1993), which recommend assessing both high-end costs and low-end costs. The high-end costs adopting the OSCE amounted to € 145.23 per student, while the low-end costs were € 31.51 per student. Considering the economic crisis and the cost-containment measures applied also in nursing education, strategies for further reducing costs are discussed.     

The usefulness and acceptance of the OSCE in nursing schools

This qualitative study explores the usefulness and acceptance attributed by students and faculty members to an Objective Structured Clinical Evaluation (OSCE) administered to nursing undergraduates in Catalonia (Spain) for 10 years. Seventy undergraduate nursing students and twelve faculty members participated in the study. The data collection techniques included an open-ended questionnaire, a student focus group, and individualized faculty interviews. The students experienced the OSCE positively as a learning event that offered an opportunity for feedback that could help them master the required competencies. The OSCE increased students' responsibility by presenting them with a set of challenges that they had to tackle individually. Moreover, it reaffirmed their confidence in situations that closely resembled professional practice. Faculty members valued the ability of the OSCE to integrate and assess competencies, its objectivity, and the indirect information it provided on the effectiveness of the curriculum. The educational impact attributed to the OSCE and its acceptance among faculty and students suggest that it would be useful to re-implement it in the Bachelor's of Nursing in Catalan universities. Our findings may be of use to other nursing programs considering how to assess competency-based education, especially in the context of the European Higher Education Area.     

An objective structured clinical exam in a multisite obstetrics and gynecology clerkship

An Objective Structured Clinical Exam (OSCE) was developed to test students completing a multiple site clinical clerkship. Common ambulatory problems were emphasized. The exam was given in the outpatient department where 60 students rotated through approximately 10 short problem stations. Gynecologic teaching associates, clinical staff, and others served as simulated patients. The specific stations, exam logistics, and expenses are explained. OSCEs can consistently and specifically test important clinical skills. Subjectivity, variability, and lack of structure in many evaluation systems support the use of this and other objective tools. OSCE performances were compared at different sites and to other evaluation measures. The OSCE correlated with clinical performance evaluations, but not with the National Board of Medical Examiners Subtest. Development of the exam also necessitated clarification of learning objectives for the faculty. Implementation provided insight into curriculum deficiencies.     

Novice Nurse Practitioner Faculty and the Online Educator’s Role

Nursing programs continue to transition or establish online education programs in response to the need for nursing faculty to teach in postlicensure to higher educational degree programs. Recruitment efforts target experienced nurse practitioners (NPs) in clinical practice who may be considering an academic role. Before the acceptance of the novice NP faculty role, the components of the role, faculty development resources, extent of orientation, and role expectations must be discussed. Limited preparation to online education environments jeopardizes the recruitment and retention of qualified NP faculty. This article reviews strategies for successfully acclimating novice faculty to online teaching.