The Objective Structured Clinical Examination (OSCE) as a strategy for assessing clinical competence in midwifery education in Ireland: a critical review

In Ireland, to register as a midwife, all student midwives must be deemed competent to practice with the assessment of competence an essential component of midwifery education. A variety of assessment strategies, including observed practice, clinical interviews, portfolios of reflection, the Objective Structured Clinical Examination (OSCE) and written examination papers, are utilised to assess midwifery students' clinical competence. In this paper, a critical review of the OSCE as a strategy for assessing clinical competence in one third level institution in Ireland is offered. Although utilised for assessing competence across a range of areas (e.g. obstetric emergencies and pharmacology/drug administration), the use of the OSCE for assessing midwifery students' competence in lactation and infant feeding practices, as an example for this paper, is described. The advantages, disadvantages, validity and reliability of the OSCE, as an assessment strategy, are critically explored. Recognising that no single assessment strategy can provide all the information required to assess something as complex as clinical performance, the OSCE, when viewed alongside other forms of assessment, and with relevance to the topic under examination, may be considered a valuable strategy for enhancing the assessment of students' clinical competence, and for embracing diversity within midwifery education and training.     

Objective structured clinical examination (OSCE): review of literature and implications for nursing education

Objective structured clinical examination (OSCE) has been used to assess medical students since the mid 1970s, and in more recent years has been increasingly utilised to assess students from nursing and the allied health professions. This growing utilisation has led to considerable debate within the literature pertaining to the optimal use of OSCE as an assessment process. The purpose of this paper is to present a narrative review regarding some of the key issues affecting the utilisation of OSCE within the assessment of nursing students. The paper briefly reviews the historical development of OSCE within health professional assessment, and summarises some of its key strengths and limitations. It then offers a more 'in depth' consideration of the research literature pertaining to the reliability and validity of the OSCE process, which is then used as a basis for exploring some of the particular issues that need to be considered when OSCE is used to assess nursing students. Key issues identified include the need to carefully prepare and pilot new OSCE examinations and marking tools in order to ensure reliability and validity is optimised, and also the need to carefully consider the length, number and interdependence of OSCE stations to ensure that the potentially competing requirements of validity and reliability are balanced. The paper also recognises that whilst the evidence base regarding OSCE is extensive, the evidence base specific to nursing is more limited. There is therefore scope for further research in this area, as well as the need for careful debate regarding how national guidance may be a way of enhancing and standardising future OSCE examinations. The paper concludes that whilst caution must be applied in relying on OSCE as a sole means of practitioner assessment, used carefully it can make a helpful and meaningful contribution to health professional education.     

The development of a six-station OSCE for evaluating the clinical competency of the student nurses before graduation: A validity and reliability analysis

BackgroundAssessing clinical competency in undergraduate nursing students is essential before they enter clinical practice. The objective structured clinical examination (OSCE) is widely used and is part of clinical competency assessment.AimThe aim of this study was to develop and validate a six-station OSCE for evaluating the clinical competency of the student nurses before graduation.MethodsOne-group longitudinal observational study. An OSCE of 6 stations of 10 min each was adopted. Blueprint setting, checklist development, and standard patient and examiner training were conducted. The mean, standard deviation, item scale (item-total correlation) and comparisons of extreme groups for item analysis. The inter-coder of examiners and inter-case of students were used to test reliability. Content Validity and concurrent validity were used to evaluate validity of OSCE.ResultsOne hundred nursing students participated in this study. The mean OSCE score of students who passed the registered nurses' examination was significantly higher than that of students who failed. Significant differences in OSCE score between students who remain versus left clinical job were observed at 3, 6 and 9 months after graduation.ConclusionThe OSCE is predictable method for assessing undergraduate students' first year clinical retention.     

Mental health and learning disability nursing students' perceptions of the usefulness of the objective structured clinical examination to assess their competence in medicine administration

The aim of this study was to evaluate mental health and learning disability nursing students' perceptions of the usefulness of the objective structured clinical examination (OSCE) in assessing their administration of medicine competence. Learning disability (n = 24) and mental health (n = 46) students from a single cohort were invited to evaluate their experience of the OSCE. A 10‐item survey questionnaire was used, comprising open‐ and closed‐response questions. Twelve (50%) learning disability and 32 (69.6%) mental health nursing students participated. The OSCE was rated highly compared to other theoretical assessments; it was also reported as clinically real and as a motivational learning strategy. However, it did not rate as well as clinical practice. Content analysis of written responses identified four themes: (i) benefits of the OSCE; (ii) suggestions to improve the OSCE; (iii) concern about the lack of clinical reality of the OSCE; and (iv) OSCE‐induced stress. The themes, although repeating some of the positive statistical findings, showed that participants were critical of the university setting as a place to conduct clinical assessment, highlighted OSCE‐related stress, and questioned the validity of the OSCE as a real‐world assessment. The OSCE has an important role in the development of student nurses' administration of medicine skills. However, it might hinder their performance as a result of the stress of being assessed in a simulated environment.     

Simulation as a high stakes assessment tool in emergency medicine

The Australasian College for Emergency Medicine (ACEM) will introduce high stakes simulation‐based summative assessment in the form of Objective Structured Clinical Examinations (OSCEs) into the Fellowship Examination from 2015. Miller's model emphasises that, no matter how realistic the simulation, it is still a simulation and examinees do not necessarily behave as in real life. OSCEs are suitable for assessing the CanMEDS domains of Medical Expert, Communicator, Collaborator and Manager. However, the need to validate the OSCE is emphasised by conflicting evidence on correlation with long‐term faculty assessments, between essential actions checklists and global assessment scores and variable interrater reliability within individual OSCE stations and for crisis resource management skills. Although OSCEs can be a valid, reliable and acceptable assessment tool, the onus is on the examining body to ensure construct validity and high interrater reliability.     

Innovation in the assessment of nursing theory and its evaluation: a team approach

In response to recent national changes in United Kingdom (UK) nurse education (e.g. devolution of assessment, moves to higher education, revision of the aims) and to local concerns (e.g. fairness to students, validity and reliability of written assessments, helping staff with less experience of assessment, student learning) an initiative has been developed at Southampton based on a team approach to marking and moderating. A five-stage evaluation was designed to accompany implementation of the initiative. The evaluation, carried out by a lecturer and an independent educational evaluator, involved both tutors/lecturers and students. Interviews, questionnaires and observation methods were used. Benefits of the initiative and of the particular model of evaluation included: increased knowledge and confidence in the validity and reliability of the marking and moderating process undertaken by tutor-teams; increased fairness to students; in-service tutor training related to student assessment; knowledge that assessment-promoted learning was taking place. A review of the total assessment programme was an unexpected outcome, including a review of the frequency and timing of assessments and of the written guidelines. The five-stage evaluation developed a feeling of involvement and heightened self-knowledge. Curricular understanding also increased; this helped to achieve the initiative as designed and intended. We recommend this model of evaluation; it promotes involvement of all concerned, students as well as staff, and generates valuable process-knowledge. It can be used in pre- and post-registration nurse education.     

简化的客观结构化临床考核在《急救护理学》技能考核中的应用

目的尝试在护理本科《急救护理学》课程技能考核中应用简化的客观结构化临床考核(objective structured clinical examination,OSCE)。方法在2007级护理本科《急救护理学》课程技能考核中应用OSCE考核方式,设3个站点,分别为心肺复苏、气道管理、ICU救护站,并对考核结果进行难度、区分度和信效度分析,同时应用自行设计的问卷对学生进行问卷调查。结果本轮OSCE各站点难度适中,区分度合理,站点间Cronbachα系数为0.354,OSCE技能考核成绩与理论考核成绩无相关,但OSCE的考核形式得到学生普遍认可。结论在《急救护理学》技能考核中运用简化的OSCE切实可行,但应从考核设计、实施等多方面入手,以提高考核结果的可信程度。     

Development and psychometric testing of the 10-item satisfaction with Nursing Skill Examination: Objective Structured Clinical Assessment scale

The Objective Structured Clinical Examination (OSCE) or Assessment (OSCA) has traditionally been used in disciplines such as medicine and nursing, to assess students' competence to perform clinical skills safely in a simulated hospital environment. Despite its accepted use, a validated and reliable tool has yet to be developed and tested to assess students' perception of and satisfaction with this mode of assessment. This study developed and tested the psychometric properties of a brief Objective Structured Clinical Examination tool for assessing student perception that could have transferability across health education settings. The study used a cross-sectional survey design. Final year students (n = 727) enrolled in an undergraduate nursing program in Western Sydney completed the 10-item Satisfaction with Nursing Skill Examination: Objective Structured Clinical Assessment (SINE-OSCA) Scale in 2017. Exploratory factor analysis uncovered a one-component structure with component loading that ranged from 0.45 to 0.86. Cronbach's alpha of the SINE-OSCA was 0.91. Socio-demographic group comparisons revealed that respondents who were: i) male (p = 0.003); ii) non-native-born (p < 0.001); iii) non-English-speaking (p < 0.001); and iv) International (p = 0.001), reported higher satisfaction with clinical assessments, as measured by the SINE-OSCA scale. The SINE-OSCA scale demonstrates validity and reliability in identifying students who may have difficulty with this mode of clinical skill assessment.     

助产专业本科助产技能综合实训课程的设置与实践

目的探讨助产专业本科层次助产技能综合实训课程的效果评价。方法以临床工作过程为主线,设计助产技能综合实训课程(44学时)。采用课堂讲授、视频教学、小组讨论、实操演练等多种形式,对2016级大学三年级30名助产专业学生进行教学。课后采用多站式客观结构化临床考试评价学生的综合实践能力,并收集学生和教师对实训课程教学的评价。结果学生操作考核成绩为(92.21±2.01)分。学生及教师对课程的内容设置、教学方法等均比较满意,认为课程可以提高助产技能的综合应用能力。结论助产技能综合实训课程有助于提高学生助产技能综合应用能力、言语表达和沟通能力、分析和解决问题的能力、合作意识,缩短实验教学与临床实践的差距。     

The validity and reliability of methods to assess the competence to practise of pre-registration nursing and midwifery students.

This paper reports findings from a study funded by the National Board for Nursing, Midwifery and Health Visiting for Scotland to test selected nursing and midwifery clinical competence assessment tools for reliability and validity.The study, which took place over two years from July 1997, involved comparing items in the selected tools with statutory competencies for nurses and midwives, collecting assessment data from a sample of 257 nursing and 43 midwifery students in four educational institutions and administering additional assessment measures (the Nursing Competencies Questionnaire (NCQ) (Bartlett et al., An evaluation of pre-registration nursing education: a literature review and comparative study of graduate outcomes, Oxford Centre for Health Care Research & Development, Oxford Brookes University, Oxford, 1998) and the Key Areas Assessment Instrument-KAAI) to the total student sample (and to their lecturers and practice assessors) at two time points which were six months apart. Our focus was the programme-specific clinical competence assessment tools but by testing these tools we also provide evidence on the validity of other methods of competence assessment.Validity of the methods was assessed, primarily, by calculating multivariate and univariate correlation coefficients between them. The NCQ and KAAI were analysed for internal consistency.The NCQ and the versions of KAAI for lecturers and practice assessors were found to have good internal consistency. The version of the KAAI tool developed for students showed reasonable internal consistency for nursing students, but less consistency for midwifery students. Correlational analysis of data collected on students showed that there is little or no relationship between most of the clinical competence assessment methods in current use, or between these methods and those introduced by the research team. This finding supports previous research, particularly in medical education and confirms that the different methods address different abilities.A clear finding from this study is that no single method is appropriate for assessing clinical competence. A multi-method UK-wide strategy for clinical competence assessment for nursing and midwifery is needed if we are to be sure that assessment reveals whether or not students have achieved the complex repertoire of knowledge, skills and attitudes required for competent practice.     

Finding reality: the use of objective structured clinical examination (OSCE) in the assessment of mental health nursing students interpersonal skills

This paper presents the implementation of an objective structured clinical examination (OSCE) in the assessment of mental health nursing students' interpersonal skills. It begins by providing a rationale for the use of this instrument to assess such ski lls and offers a brief discussion of the development of OSCEs. The preparation and implementation of the OSCE is explored and both students' and tutors' reflections of the process are highlighted. The strengths and problems, particularly the use of an ac tor and video tape recordings are examined, in the light of other studies. The paper concludes by advocating the use of such an assessment tool as a formative exercise.     

Assessing the Validity of a Multidisciplinary Mini-Clinical Evaluation Exercise

Construct: The purpose of this study was to provide validity evidence for the mini-clinical evaluation exercise (mini-CEX) as an assessment tool for clinical skills in the workplace. Background: Previous research has demonstrated validity evidence for the mini-CEX, but most studies were carried out in internal medicine or single disciplines, therefore limiting generalizability of the findings. If the mini-CEX is to be used in multidisciplinary contexts, then validity evidence should be gathered in similar settings. The purpose of this study was to gather further validity evidence for the mini-CEX but in a broader context. Specifically we sought to explore the effects of discipline and rater type on mini-CEX scores, internal structure, and the relationship between mini-CEXs and OSCEs in a multidisciplinary context. Approach: During clerkship, medical students completed eight different rotations (family medicine, internal medicine, surgery, psychiatry, pediatrics, emergency, anesthesiology and obstetrics and gynecology). During each rotation, mini-CEX forms and a written examination were completed. Two multidisciplinary OSCEs (in Clerkship Year 3 and start of Year 4) assessed clinical skills. The reliability of the mini-CEX was assessed using Generalizability analyses. To assess the influence of discipline and rater type, mean scores were analyzed using a factorial analysis of variance. The total mini-CEX score was correlated to scores from the students' respective OSCEs and corresponding written exams. Results: Eighty-two students met inclusion criteria for a total of 781 ratings (average of 9.82 mini-CEX forms per student). There was a significant effect of discipline (p < .001, = .16), and faculty provided lower scores than nonfaculty raters (7.12 vs. 7.41; p = .002, = .02). The g-coefficient was .53 when discipline was included as a facet and .23 when rater type was a facet. There were low, but statistically significant correlations between the mini-CEX and scores for the 4th-year OSCE Total Score and the OSCE communication scores, r(80) = .40, p < .001 and r(80) = .29, p = .009. The mini-CEX was not correlated with the written examination scores for any of the disciplines. Conclusions: Our results provide conflicting findings for validity evidence for the mini-CEX. Mini-CEX ratings were correlated to multidisciplinary OSCEs but not written examinations, supporting the validity argument. However, reliability of the mini-CEX was low to moderate, and error accounted for the greatest amount of variability in scores. There was variation in scores due to discipline and resident raters gave higher scores than faculty. These results should be considered when considering the use of the mini-CEX in different contexts.     

Comparison of the effectiveness of teaching strategies for a pediatric pain management program for undergraduate nursing students: A quantitative evaluation using an objective structured clinical examination

Pain is a common experience for hospitalized children; however, nursing students are often not adequately trained in pediatric pain management. Innovative teaching strategies, such as e-learning, have been employed for instructing students, however success of these platforms has not been quantitatively measured. This study compared students' knowledge and skill performance following a researcher-designed pain management program administered with three teaching strategies: traditional face-to-face, e-learning, or blended learning. Undergraduate nursing students in Taiwan (N = 296) randomly assigned to one of the teaching strategies participated. Knowledge of pain management, and pain management skills were quantified. A subjective assessment of attitudes towards learning found no significant difference between groups. Knowledge of pediatric pain management did not differ significantly between groups (p = 0.36). A 15-item objective structured clinical examination (OSCE) measured the competency of pain management skills; scores were not significantly different, regardless of teaching strategy (p = 0.70). Traditional face-to-face teaching was as effective as both innovative strategies. Evaluating students’ skills following a pain management program with an OSCE provided a quantitative assessment of competency. Innovative strategies for teaching pediatric pain management could be a cost-effective way to provide flexible learning opportunities for nursing students who are distant from educational institutions.     

Standardization of Clinical Skill Evaluation in Physical/Occupational Therapist Education –Effects of Introduction of an Education System Using OSCE–

[Purpose] A major issue in physical/occupational therapist education is the improvement of students'' clinical techniques. In this study, we introduced an education system using an Objective Structured Clinical Examination (OSCE), and made an attempt at standardization of its evaluation. [Subjects] The subjects were 227 students in the classes of 2008 to 2010 who enrolled at our university between 2004 and 2006, before the introduction of the education system using OSCE, and 221 students in the classes of 2011 to 2013 who enrolled between 2007 and 2009, after the introduction. [Methods] Performances in attitude and skills (performance in clinical training and OSCE) were compared between before and after the introduction of OSCE. OSCE results were compared between before and after clinical trainings at each OSCE Level; and the correlation of between performances in clinical training and OSCE was examined. [Results] Performances in OSCE and clinical training (attitude, skills) were improved by the introduction of the education system using OSCE, but no significant correlation was observed in the relationship between performances in OSCE and clinical training. [Conclusion] Further studies should be conducted aiming at the standardization of clinical skill evaluation in postgraduate education to establish an education system using OSCE.Key words: OSCE, Clinical skill evaluation, Standardizing