Pharmacy Students’ Perceptions and Performance on the Use of an Online Virtual Experience Tool for Practicing Objective Structured Clinical Examinations

Objective. To examine pharmacy students’ performance on and perceptions regarding the use of an interactive online tool for practicing to take objective structured clinical examinations (OSCEs).Methods. The Monash OSCE Virtual Experience (MOVE), an online module consisting of 20 pharmacy case scenarios with virtual patients, was piloted with final-year pharmacy students at Monash University campuses in Australia and Malaysia. A mixed methods approach that included reviewing user attempts and comparing grades, collecting student-administered questionnaires, and holding focus groups was used to examine students’ perception and performance.Results. More than 99% of all students attempted at least one online case scenario in preparation for their final in-person OSCE, and 81% attempted all 20 scenarios two or more times. Ninety percent of students at the Malaysia campus and 70% of students at the Australia campus reported that MOVE was a helpful study tool for their OSCE preparation. However, a raw comparison of user attempts and OSCE grades did not find a direct correlation between online module attempts and assessment grades. Self-administered questionnaire and focus group results indicated that MOVE prepared students for targeted and time-restricted history-taking and problem-solving skills. Overall, students perceived MOVE to be a useful learning tool and a less overwhelming learning experience than were face-to-face sessions. Nevertheless, students still preferred face-to-face OSCE practice with simulated patients over online practice with virtual patients.Conclusion. The Monash OSCE Virtual Experience was perceived by our students as a flexible and useful online learning aid in preparing for their final-year OSCE However, there was no direct correlation between online practice attempts and students’ exam grades.     

Systematic Review of Predictors of Success for the North American Pharmacist Licensure Examination

Objective. Studies have examined possible predictors of success on the North American Pharmacist Licensure Examination (NAPLEX). This systematic review investigated the literature on potential predictors of success on the NAPLEX.Findings. Articles were included in the review if they studied student characteristics and academic performance as independent variables and NAPLEX scores and/or pass rates as an outcome. Data were extracted from each article for students’ demographics or variables, sample size, methods of statistical analyses, and results reporting correlation or predictability. From 40 articles retrieved from the initial search and sorting, 20 studies were included in the final review per inclusion criteria. Three studies included all the pharmacy programs, 15 were single-institution studies, two were multi-institution studies, and four had been published as posters. Among 30 different variables identified as potential predictors of success on the NAPLEX, the most examined variables were student age at matriculation, having a prior degree, Pharmacy College Admission Test (PCAT) scores, cumulative pharmacy school grade point average (GPA), overall Pharmacy Curriculum Outcomes Assessment (PCOA) scores, and PCOA content areas scores. Positively correlated factors included PCAT scores, not having a prior degree, prepharmacy and pharmacy school GPA, institutional characteristics, and PCOA scores. Negatively correlated factors included older age at admission.Summary. Cumulative pharmacy school GPA and PCOA scores were predictors of NAPLEX success consistently in the studies. The effects of preadmission student characteristics on NAPLEX success varied and were not consistently correlated or predictive.     

Pharmacy Student Engagement,Performance, and Perception in a Flipped Satellite Classroom

Objective. To determine whether “flipping” a traditional basic pharmaceutics course delivered synchronously to 2 satellite campuses would improve student academic performance, engagement, and perception.Design. In 2012, the basic pharmaceutics course was flipped and delivered to 22 satellite students on 2 different campuses. Twenty-five condensed, recorded course lectures were placed on the course Web site for students to watch prior to class. Scheduled class periods were dedicated to participating in active-learning exercises. Students also completed 2 course projects, 3 midterm examinations, 8 graded quizzes, and a cumulative and comprehensive final examination.Assessment. Results of a survey administered at the beginning and end of the flipped course in 2012 revealed an increase in students’ support for learning content prior to class and using class time for more applied learning (p=0.01) and in the belief that learning key foundational content prior to coming to class greatly enhanced in-class learning (p=0.001). Significantly more students preferred the flipped classroom format after completing the course (89.5%) than before completing the course (34.6%). Course evaluation responses and final examination performance did not differ significantly for 2011 when the course was taught using a traditional format and the 2012 flipped-course format. Qualitative findings suggested that the flipped classroom promoted student empowerment, development, and engagement.Conclusion. The flipped pharmacy classroom can enhance the quality of satellite students’ experiences in a basic pharmaceutics course through thoughtful course design, enriched dialogue, and promotion of learner autonomy.     

A Concept Mapping Activity to Enhance Pharmacy Students’ Metacognition and Comprehension of Fundamental Disease State Knowledge

Objective. To examine the impact of pre-class concept mapping activities on pharmacy students'' ability to self-assess their degree of foundational disease state knowledge and predict their pre-class quiz performance.Methods. Second year pharmacy students in a problem-based learning course were responsible for self-directed learning of foundational knowledge for 14 disease states. After completing their independent pre-class reading, students worked in groups to create concept maps for which feedback was provided for four laboratory sessions, worked in groups to create concept maps but received no formal feedback for three laboratory sessions, and did not engage in any formal group activity for seven laboratory sessions. The day following each session, prior to the formal in-class discussion, students were asked to predict the number of questions they could answer correctly on a quiz covering foundational knowledge and then completed the quiz. Quiz performance was compared based on the three conditions, and bias and absolute bias were calculated to evaluate students’ metacognitive skills.Results. There was no difference in pharmacy students’ metacognition based on the conditions, as reflected by inaccuracy between predicted and actual quiz scores. However, when students had engaged in concept mapping the previous day, their quiz performance was significantly higher than when they had not.Conclusion. Concept mapping did not improve pharmacy students’ metacognitive skills but did have a small effect on their quiz performance. More research is needed to tease apart the roles of concept mapping, group activity, and feedback in altering pharmacy students’ quiz performance and metacognitive skills.     

The Impact of Two Longitudinal Professionalism Courses on Student Pharmacists’ Empathy

Objective. To determine whether empathy increased in first-year student pharmacists after completing longitudinal professionalism courses at two schools of pharmacy, identify potential moderators, and assess whether students’ conceptualization of empathy changed with time.Methods. Surveys to assess empathy and other variables were administered to student pharmacists at baseline and at the end of two professionalism courses. Baseline and follow-up scores were compared to detect changes over time. Multivariable analysis was used to identify predictors of empathy scores. Factor analysis was performed to ascertain changes in the dimensionality of empathy.Results. Students’ demographics and baseline empathy scores differed between the two schools. Predictors of empathy at baseline included age, female gender, prior health care experience, and altruism score. A small increase in empathy was observed at one school but not in the combined cohort. Empathy was more likely to increase among female students, those with less health care experience, and those who did not work during the school year. Factor analyses suggested that students’ conceptual clarity about empathy improved over time and became more consistent with existing models.Conclusion. Although an increase in empathy was not observed in the overall cohort, subgroups of students who may derive greater benefit from empathy-related interventions were identified. Factor analyses suggested that students’ conceptual understanding of empathy improved, representing a potential alternative outcome assessment for affective domains. Given differences in demographics, instructional methodologies, and changes in empathy at each school, this study reinforces the importance of replication and multicenter studies to understand the generalizability of educational research.     

Evaluation of Clinical and Communication Skills of Pharmacy Students and Pharmacists with an Objective Structured Clinical Examination

Objective. To evaluate how effectively pharmacy students and practicing pharmacists communicate and apply knowledge to simulations of commonly encountered patient scenarios using an objective structured clinical examination (OSCE).Design. Second-, third-, and fourth-year pharmacy students completed an OSCE as part of their required courses in 2012 and 2013. All students in both years completed identical OSCE cases. Licensed pharmacists were recruited to complete the OSCE and serve as controls in 2012. A survey assessed student perception and acceptance of the OSCE as well as student confidence in performance.Assessment. Licensed pharmacists had significantly higher clinical and communication skills scores than did pharmacy students. Student progression in communication and clinical skills improved significantly over time. Survey results indicated that students felt the OSCE was well-structured and assessed clinical skills taught in pharmacy school; 86% of students felt confident they could provide these skills.Conclusion. Objective structured clinical examinations can evaluate clinical competence and communication skills among professional students. Implementation of OSCEs may be an effective tool for assessment of the Center for the Advancement of Pharmacy Education domains.     

Team-Based Learning in US Colleges and Schools of Pharmacy

Objective. To characterize the use of team-based learning (TBL) in US colleges and schools of pharmacy, including factors that may affect implementation and perceptions of faculty members regarding the impact of TBL on educational outcomes.Methods. Respondents identified factors that inhibit or enable TBL use and its impact on student learning. Results were stratified by type of institution (public/private), class size, and TBL experience.Results. Sixty-nine of 100 faculty members (69%) representing 43 (86%) institutions responded. Major factors considered to enable TBL implementation included a single campus and student and administration buy-in. Inhibiting factors included distant campuses, faculty resistance, and lack of training. Compared with traditional lectures, TBL is perceived to enhance student engagement, improve students’ preparation for class, and promote achievement of course outcomes. In addition, TBL is perceived to be more effective than lectures at fostering learning in all 6 domains of Bloom’s Taxonomy.Conclusions. Despite potential implementation challenges, faculty members perceive that TBL improves student engagement and learning.     

Objective Structured Clinical Examinations (OSCEs) compared with traditional assessment methods

Objectives. To compare objective structured clinical examinations (OSCEs) and traditional assessment methods among recent pharmacy graduates.Methods. Individual student performance in OSCEs was compared with performance on traditional pharmacy-practice examinations at the same level of program study.Results. A moderate correlation was found between individual attainment in OSCE examinations and on traditional pharmacy practice examinations at the same level.Conclusions. OSCEs add value to traditional methods of assessment because the 2 evaluation methods measure different competencies.     

A Standardized Patient Counseling Rubric for a Pharmaceutical Care and Communications Course

Objective. To restructure a required pharmaceutical care and communications course to place greater emphasis on communication skills and include a high-stakes assessment.Design. A standardized counseling rubric was developed for use throughout the pharmacy curriculum and the counseling laboratory practicals were changed to high-stakes assessments.Assessment. An annual mid-semester and end-of-semester high-stakes patient-counseling objective structured clinical examination (OSCE) conducted prior to and after revision of the course and counseling rubric documented improvements in students’ scores. Performance on the post-course annual assessment patient counseling OSCE improved compared to that on the pre-course (p<0.001).Conclusion. The 2010 course revision improved students’ medication counseling abilities and readiness to practice. Major course revisions should be undertaken only after input from all stakeholders and with data to support the need for change.     

Test-Enhanced Learning in an Immunology and Infectious Disease Medicinal Chemistry/Pharmacology Course

Objective. To develop a series of active-learning modules that would improve pharmacy students’ performance on summative assessments.Design. A series of optional online active-learning modules containing questions with multiple formats for topics in a first-year (P1) course was created using a test-enhanced learning approach. A subset of module questions was modified and included on summative assessments.Assessment. Student performance on module questions improved with repeated attempts and was predictive of student performance on summative assessments. Performance on examination questions was higher for students with access to modules than for those without access to modules. Module use appeared to have the most impact on low performing students.Conclusion. Test-enhanced learning modules with immediate feedback provide pharmacy students with a learning tool that improves student performance on summative assessments and also may improve metacognitive and test-taking skills.     

A Situational Judgment Test to Assess Students’ Achievement of Accreditation Council for Pharmacy Education Standards 3 and 4

Objective. For pharmacy students to successfully meet competencies related to Accreditation Council for Pharmacy Education Standards 3 and 4, it is essential for pharmacy programs to assess student progression in the affective domain. The purpose of this study was to develop and assess the validity of a situational judgment test (SJT) for evaluating student mastery of Standards 3 and 4.Methods. A multi-institutional faculty team developed an 18-item SJT that consisted of scenarios asking the respondent to rank the effectiveness of four response options mapped to Standards 3 and 4. The research team systematically reviewed the literature, created items, and deliberated until consensus was achieved. Subject matter experts (SMEs) reviewed and provided feedback on the instrument. Students from two institutions were recruited to participate in cognitive interviews about the finalized instrument. Cognitive interview data were analyzed to identify themes.Results. After edits were made to the instrument based on SME feedback, students (n=18) in the cognitive interviews identified item length as a concern and commented on item/response clarity and comprehensiveness. Data from the cognitive interviews were used to modify the SJT to reduce the length and clarify items. The result was two shorter versions of the instrument, both with similar mapping to all elements in Standards 3 and 4.Conclusion. Early steps in validating the SJT suggested that the instrument may be a promising tool to assess student progression in the affective domain. The SJT instrument is intended to provide evidence of student pharmacist development that occurs in the didactic, experiential, and co-curricular portions of pharmacy education. The instrument can serve as one part of a comprehensive assessment plan.     

The Impact of Eliminating Backward Navigation on Computerized Examination Scores and Completion Time

Objective. To determine whether elimination of backward navigation during an examination resulted in changes in examination score or time to complete the examination.Methods. Student performance on six examinations in which backward navigation was eliminated was compared to performance on examinations administered to pharmacy students the previous year when backwards navigation was allowed. The primary comparison of interest was change in student performance on a subset of identical questions included on both examinations. Secondary outcomes included change in total examination score and completion time.Results. No significant reduction in examination scores was observed as a result of eliminating backward navigation. The average time that students spent on a question was significantly reduced on two of the six examinations.Conclusion. Restricting pharmacy students’ ability to revisit questions previously answered (elimination of backward navigation) on an examination had no adverse effect on scores or testing time when assessed across three years of the didactic pharmacy curriculum.     

An objective structured clinical examination to assess problem-based learning

Objectives. To compare pharmacy students’ performance on an objective structured clinical examination (OSCE) to their performance on a written examination for the assessment of problem-based learning (PBL); and to determine students’ and faculty members’ perceptions of OSCEs for PBL evaluations.Design. Four OSCEs were added to the written examination to assess 4 PBL cases in a third-year pharmacotherapy course. OSCE scores were compared to written examination scores. Faculty members evaluated student performance.Assessment. OSCE performance did not correlate with the written-examination scores. Most students (≥ 75%) agreed that OSCEs reflected their learning from PBL and measured knowledge, communication, and clinical skills. A majority of faculty members (≥75%) agreed that OSCEs should be part of PBL assessment.Conclusions. Addition of an OSCE to written examinations was valued and provided a more comprehensive assessment of the PBL experience.     

Effectiveness of Providing Video Podcasts to Pharmacy Students in a Self-Study Pharmaceutical Calculations Module

Objective. To evaluate the effect of providing educational video podcasts to first-year pharmacy students enrolled in a self-taught pharmaceutical calculations module.Methods. Six video podcasts were designed and posted online for pharmacy students to review prior to taking pharmaceutical calculations quizzes over the course of one year. The video podcasts demonstrated how to solve the most difficult problems on each of the six quizzes. Quiz performance was compared for the 2017-2018 year (no videos available) and the 2018-2019 year (videos available). Students provided feedback about the videos in a year-end survey.Results. Overall quiz scores and passing rates improved during the 2018-2019 academic year. Enhanced student performance on three of the six quizzes was demonstrated. A majority of students strongly agreed that the videos were helpful.Conclusion. Watching video podcasts was effective in improving pharmacy students’ performance on calculations quizzes. The students indicated that the podcasts were helpful in improving their math skills and facilitated their learning of the material.     

Online Self-testing Resources Prepared by Peer Tutors as a Formative Assessment Tool in Pharmacology Courses

Objective. To assess the effectiveness of optional online quizzes written by peer tutors in a pharmacology course for doctor of pharmacy students.Methods. Online quizzes were written by peer tutors for second-year pharmacy students. Quizzes reflected the material taught during lecture and were in a format similar to that of the examinations. Data related to performance on each quiz and each examination were collected throughout the semester. At the end of the semester, students and peer tutors were surveyed to gather information on the utility and success of the quizzes.Results. Students taking online quizzes performed significantly better on examinations than those who did not take quizzes. In addition, students received higher scores on examinations than when practicing with the quizzes. Surveys suggest that students liked the quizzes and felt they increased their confidence and performance on examinations.Conclusion. The quizzes were beneficial to student performance on examinations as well as student perception of performance and confidence going into the examinations. Quizzes were also beneficial learning experiences for peer tutors.     

Exploring the Role of Item Scenario Features on Situational Judgment Test Response Selections

Objective. To explore pharmacists’ and pharmacy students’ perceptions regarding the significance of changing the features of test item scenario (eg, switching from a health care to a non-health care context) on their situational judgment test (SJT) responses.Methods. Fifteen Doctor of Pharmacy students and 15 pharmacists completed a 12-item SJT intended to measure empathy. The test included six scenarios in a health care context and six scenarios in a non-health care context; participants had to rank potential response options in order of appropriateness and no two items could be of equal rank. Qualitative data were collected individually from participants using think-aloud and cognitive interview techniques. During the cognitive interview, participants were asked how they selected their final responses for each item and whether they would have changed their answer if features of the scenario were switched (eg, changed to a non-health care context if the original item was in a health care context). Interviews were transcribed and a thematic analysis was conducted to identify the features of the scenario for each item that were perceived to impact response selections.Results. Participants stated that they would have changed their responses on average 51.3% of the time (range 20%-100%) if the features of the scenario for an item were changed. Qualitative analysis identified four pertinent scenario features that may influence response selections, which included information about the examinee, the actors in the scenario, the relationship between examinee and actors, and details about the situation. There was no discernible pattern linking scenario features to the component of empathy being measured or participant type.Conclusion. Results from this study suggest that the features of the scenario described in an SJT item could influence response selections. These features should be considered in the SJT design process and require further research to determine the extent of their impact on SJT performance.     

Impact of Hybrid Delivery of Education on Student Academic Performance and the Student Experience

Objectives

To compare student academic performance and the student experience in the first-year doctor of pharmacy (PharmD) program between the main and newly opened satellite campuses of the University of Maryland.

Methods

Student performance indicators including graded assessments, course averages, cumulative first-year grade point average (GPA), and introductory pharmacy practice experience (IPPE) evaluations were analyzed retrospectively. Student experience indicators were obtained via an online survey instrument and included involvement in student organizations; time-budgeting practices; and stress levels and their perceived effect on performance.

Results

Graded assessments, course averages, GPA, and IPPE evaluations were indistinguishable between campuses. Students'' time allocation was not different between campuses, except for time spent attending class and watching lecture videos. There was no difference between students'' stress levels at each campus.

Conclusions

The implementation of a satellite campus to expand pharmacy education yielded academic performance and student engagement comparable to those from traditional delivery methods.     

Implementation and Evaluation of a 10-Week Health Equity Curriculum for Pharmacy Students

Objective. To describe a health equity curriculum created for pharmacy students and evaluate students’ perceptions and structural competency after completion of the curriculum.Methods. A health equity curriculum based on transformative learning and structural competency frameworks was implemented as a 10-week mandatory component of the pass-no pass neuropsychiatric theme for second year pharmacy students. Each week, students reviewed materials around a neuropsychiatric-related health equity topic and responded to discussion prompts through asynchronous forums or synchronous online video discussions. The effectiveness of the health equity curriculum was evaluated through assessment of structural competency through a validated instrument, an objective structured clinical examination (OSCE), and a questionnaire.Results. All enrolled second year pharmacy students (n=124) participated in the health equity curriculum. Of the 75 students who completed the structural competency instrument, 46 (61%) were able to identify structural determinants of health, explain how structures contribute to health disparities, or design structural interventions. Ninety-six of the 124 students (77%) were able to address their OSCE standardized patient’s mistrust in the health care system. Thematic analysis of student comments elucidated three themes: allyship, peer connection, and self-awareness. Students rated asynchronous discussion forums as significantly less effective than online video discussions and patient cases for achieving curricular objectives.Conclusion. A mandatory curriculum delivered remotely throughout the didactic pharmacy curriculum using a blended learning approach was an effective way to incorporate health equity content and conversations into existing courses. Implementation of this or similar curriculums could be an important step in training pharmacy students to be advocates for social justice.     

A Capstone Course with a Comprehensive and Integrated Review of the Pharmacy Curriculum and Student Assessment as a Preparation for Advanced Pharmacy Practice Experiences

Objective. To create a capstone course that provides a comprehensive and integrated review of the pharmacy curriculum with a broad range of assessment tools to evaluate student knowledge and skills as a final preparation prior to beginning fourth-year advanced pharmacy practice experiences (APPEs).Design. The capstone course was a 4 credit-hour, case-based course. Eight comprehensive cases were assigned to students over the course of the term. The cases were designed to mimic complex clinical scenarios that students were likely to encounter during an APPE. Students were required to prepare a written and oral presentation for each case and were assessed on material covered during the cases. Faculty members presented weekly reviews on selected topics such as calculations, pharmacokinetics, and pharmaceutical compounding. At the end of the course, students took an observed structured clinical examination (OSCE), which simulated the Georgia Board of Pharmacy Practical Examination, and a comprehensive examination designed to mimic the NAPLEX (North American Pharmacy Licensure Examination).Assessment. Evaluation of student outcomes was based on written and verbal presentations of the cases, multiple-choice examinations, a short-answer calculations examination, an “Errors and Omissions” examination, a standardized patient encounter, and pharmaceutical compounding examinations. Ninety-five percent of students successfully passed the course on their first attempt. Student feedback indicated satisfaction with the depth, breadth, and organization of material covered and felt that the course helped prepare them for APPEs.Conclusion. The culminating experience of the capstone course gave students a thorough review of practical, clinical, and communication skills and provided faculty members with feedback regarding the curriculum through robust assessment.     

Training Malaysian Pharmacy Undergraduates with Knowledge and Skills on Smoking Cessation

Objective. To evaluate the feasibility of an online training module, Certified Smoking Cessation Service Provider (CSCSP), developed for practicing pharmacists to equip pharmacy students with knowledge necessary for smoking cessation counseling and to assess the changes in student knowledge and skills regarding smoking cessation following training.Design. Sixty third-year and 80 fourth-year pharmacy undergraduates (N=140) were given access to an online module, the main intervention in the study. Two linkable questionnaires were administered to assess students’ preintervention and postintervention knowledge. For the third-year students, an additional role-play training component was incorporated, and student skills were assessed during week 14 with an Objective Structured Clinical Examination (OSCE).Assessment. Preintervention and postintervention knowledge assessments were completed by 130 (92.8%) students. Sixty-six students scored above 50% for the knowledge component postintervention, compared to 13 at preintervention, demonstrating significant improvement (x2(1, N=130)=32, p=0.003). All third-year students completed the intervention, and 66.7% were able to counsel excellently for smoking cessation, scoring more than 80%.Conclusion. The CSCSP online module developed for practicing professionals was found suitable for equipping pharmacy undergraduates with knowledge on smoking cessation topics. The module, along with role-play training, also equipped students with knowledge and skills to provide smoking cessation counseling.