Relationship Between Assessment Item Format and Item Performance Characteristics

Objective

To evaluate the relationship between assessment item formats (case-based versus noncase-based) and item performance characteristics.

Methods

Assessment items (1,575) were collected from examinations administered in several therapeutics courses over 4 academic years. Items were categorized as either “case-based” or “noncase-based” and item performance characteristics (discrimination index and level of difficulty) were evaluated.

Results

Noncase-based items represented approximately three-fourths of all items that were evaluated, and demonstrated a higher discrimination index than case-based items. Case-based items were generally lengthier and included more detailed information than noncase-based items; however, they were not more difficult and exhibited a lower discrimination index. Secondary analyses revealed that 5-foil multiple-choice items are more difficult and have a higher discrimination index compared to 4-foil items.

Conclusion

The format used for an examination/test item (case-based or noncase-based) has an impact on item performance characteristics.     

Assessing the Effectiveness of an Evidence-based Practice Pharmacology Course Using the Fresno Test

Objective. To assess the effectiveness of an evidence-based practice (EBP) pharmacology elective course to teach EBP skills using the Fresno Test (FT).Methods. Pharmacy faculty members and medical librarians developed the elective course and offered it to two cohorts of doctor of pharmacy (PharmD) students. A pre/post intervention study design was used. Seven of 12 FT items were chosen to measure specific EBP skills: Ask, Access, Appraise and Apply. Pre/postcomposite and FT item mean scores were compared using Student’s t test with p<0.05 set as significant a priori.Results. Composite FT mean scores increased significantly for both cohorts. Mean scores for both cohorts increased significantly in four of the seven FT items but on different FT items.Conclusion. As a profession that commonly uses evidence-based guidelines, developing and integrating an EBP course in the PharmD curriculum is worth considering.     

Reliability of a Minimal Competency Score for an Annual Skills Mastery Assessment

Objective. To determine whether the modified Angoff process can be used to calculate a reliable minimal competency (“cut”) score for the Annual Skills Mastery Assessment (ASMA).Methods. Three panels of pharmacy faculty members used a modified Angoff method to create a minimal competency score for 60 previously used test items. The panels did not know which items were included. Data were analyzed to determine differences between rating sessions, faculty type, item difficulty, and rater scoring bias.Results. The cut score generated was not significantly different by session or faculty type. The range of cut scores varied by less than 3% per examination. Faculty panelists correctly predicted student performance on items grouped as easy, medium, and hard.Conclusion. A properly constructed faculty panel can determine a reliable cut score and accurately rank relative test item difficulty using the modified Angoff process.     

Validation of an Empathy Scale in Pharmacy and Nursing Students

Objective. To validate an empathy scale to measure empathy in pharmacy and nursing students.Methods. A 15-item instrument comprised of the cognitive and affective empathy domains, was created. Each item was rated using a 7-point Likert scale, ranging from strongly disagree to strongly agree. Concurrent validity was demonstrated with the Jefferson Scale of Empathy – Health Professional Students (JSE-HPS).Results. Reliability analysis of data from 216 students (pharmacy, N=158; nursing, N=58) showed that scores on the empathy scale were positively associated with JSE-HPS scores (p<0.001). Factor analysis confirmed that 14 of the 15 items were significantly associated with their respective domain, but the overall instrument had limited goodness of fit.Conclusions. Results of this study demonstrate the reliability and validity of a new scale for evaluating student empathy. Further testing of the scale at other universities is needed to establish validity.     

Pharmacy Student Engagement,Performance, and Perception in a Flipped Satellite Classroom

Objective. To determine whether “flipping” a traditional basic pharmaceutics course delivered synchronously to 2 satellite campuses would improve student academic performance, engagement, and perception.Design. In 2012, the basic pharmaceutics course was flipped and delivered to 22 satellite students on 2 different campuses. Twenty-five condensed, recorded course lectures were placed on the course Web site for students to watch prior to class. Scheduled class periods were dedicated to participating in active-learning exercises. Students also completed 2 course projects, 3 midterm examinations, 8 graded quizzes, and a cumulative and comprehensive final examination.Assessment. Results of a survey administered at the beginning and end of the flipped course in 2012 revealed an increase in students’ support for learning content prior to class and using class time for more applied learning (p=0.01) and in the belief that learning key foundational content prior to coming to class greatly enhanced in-class learning (p=0.001). Significantly more students preferred the flipped classroom format after completing the course (89.5%) than before completing the course (34.6%). Course evaluation responses and final examination performance did not differ significantly for 2011 when the course was taught using a traditional format and the 2012 flipped-course format. Qualitative findings suggested that the flipped classroom promoted student empowerment, development, and engagement.Conclusion. The flipped pharmacy classroom can enhance the quality of satellite students’ experiences in a basic pharmaceutics course through thoughtful course design, enriched dialogue, and promotion of learner autonomy.     

Student Engagement in Pharmacology Courses Using Online Learning Tools

Objective. To assess factors influencing student engagement with e-tools used as a learning supplement to the standard curriculum in pharmacology courses.Design. A suite of 148 e-tools (interactive online teaching materials encompassing the basic mechanisms of action for different drug classes) were designed and implemented across 2 semesters for third-year pharmacy students.Assessment. Student engagement and use of this new teaching strategy were assessed using a survey instrument and usage statistics for the material. Use of e-tools during semester 1 was low, a finding attributable to a majority (75%) of students either being unaware of or forgetting about the embedded e-tools and a few (20%) lacking interest in accessing additional learning materials. In contrast to semester 1, e-tool use significantly increased in semester 2 with the use of frequent reminders and announcements (p<0.001).Conclusion. The provision of online teaching and learning resources were only effective in increasing student engagement after the implementation of a “marketing strategy” that included e-mail reminders and motivation.     

Factors Associated With Health-Related Quality of Life of Student Pharmacists

Objective. To assess the health-related quality of life (HRQoL) of student pharmacists and explore factors related to HRQoL outcomes of student pharmacists in a doctor of pharmacy (PharmD) program at a public university.Methods. A survey instrument was administered to all student pharmacists in a PharmD program at a public university to evaluate differences and factors related to the HRQoL outcomes of first-year (P1), second-year (P2), third-year (P3), and fourth-year (P4) student pharmacists in the college. The survey instrument included attitudes and academic-related self-perception, a 12-item short form health survey, and personal information components.Results. There were 304 students (68.6%) who completed the survey instrument. The average health state classification measure and mental health component scale (MCS-12) scores were significantly higher for P4 students when compared with the P1through P3 students. There was no difference observed in the physical component scale (PCS-12) scores among each of the 4 class years. Significant negative impact on HRQoL outcomes was observed in students with higher levels of confusion about how they should study (scale lack of regulation) and concern about not being negatively perceived by others (self-defeating ego orientation), while school satisfaction increased HRQoL outcomes (SF-6D, p<0.001; MCS-12, p=0.013). A greater desire to be judged capable (self-enhancing ego-orientation) and career satisfaction were positively associated with the PCS-12 scores (p<0.05).Conclusion. Factors associated with the HRQoL of student pharmacists were confusion regarding how to study, ego orientation, satisfaction with the chosen college of pharmacy, and career satisfaction. First-year through third-year student pharmacists had lower HRQoL as compared with P4 students and the US general population. Support programs may be helpful for students to maintain or improve their mental and overall health.     

Impact of Student- Versus Instructor-Directed Case Discussions on Student Performance in a Pharmacotherapy Capstone Course

Objective. To evaluate the impact of incorporating student-directed (SD) vs instructor-directed (ID) active learning on student performance in a pharmacotherapy capstone course.Design. This 9-credit course was redesigned from exclusively ID case discussions to a format in which half were SD and half were ID. Student performance on evaluation questions derived from SD sessions was compared with that from ID sessions.Assessment. Overall, students (n=299) performed better on ID-session questions than on SD-session questions (78.7% vs 75.3%, correctly answered, respectively; p<0.001). For written evaluations, students performed better on ID-session questions than on SD-session questions (79.8% vs 73.9%, respectively; p<0.001). For verbal evaluations, students performed better on SD-session questions than on ID-session questions (79.5% vs 74.5%, respectively; p<0.001). After the course revision, student confidence regarding their ability to think critically, solve problems, make decisions, and pursue lifelong learning was high, and student and faculty feedback was positive.Conclusion. Student performance in a pharmacotherapy capstone course remained acceptable when a combination of SD and ID active learning was used, but the addition of SD learning did not translate to better performance on course evaluations.     

Correlation of the Health Sciences Reasoning Test With Student Admission Variables

Objectives. To assess the association between scores on the Health Sciences Reasoning Test (HSRT) and pharmacy student admission variables.Methods. During the student admissions process, cognitive data, including undergraduate grade point average and Pharmacy College Admission Test (PCAT) scores, were collected from matriculating doctor of pharmacy (PharmD) students. Between 2007 and 2009, the HSRT was administered to 329 first-year PharmD students. Correlations between HSRT scores and cognitive data, previous degree, and gender were examined.Results. After controlling for other predictors, 3 variables were significantly associated with HSRT scores: percentile rank on the reading comprehension (p<0.001), verbal (p<0.001), and quantitative (p<0.001) subsections of the PCAT.Conclusions. Scores on the reading comprehension, verbal, and quantitative sections of the PCAT were significantly associated with HSRT scores. Some elements of critical thinking may be measured by these PCAT subsections. However, the HSRT offers information absent in standard cognitive admission criteria.     

Impact of Required Versus Optional Remake of a Preparation on Pharmacy Students’ Compounding Accuracy

Objective. This retrospective study investigated the impact of a required vs an optional remake requirement on student performance in a compounding laboratory course in which students’ compounded preparations were analyzed.Methods. The analysis data for several preparations made by students over a 3-year period were compared for differences in the analyzed content of the active principal ingredient and the number of students who successfully compounded the preparation on the first attempt.Results. Students’ compounding accuracy was significantly better for the ketoprofen (pluronic lecithin organogel [PLO]) emulsion (p= 0.003) and mock co-enzyme Q10 troches (p< 0.001) when remaking an inaccurate preparation was optional rather than required. There were no significant differences in the parameters for the other compounded preparations.Conclusion. Student performance did not decrease when students were given the option to remake an inaccurate preparation. Factors such as the difficulty of the preparation, time spent compounding, and impact on the student’s final course grade also may have influenced student performance.     

A Five-Year Evaluation of Examination Structure in a Cardiovascular Pharmacotherapy Course

Objective. To evaluate the composition and effectiveness as an assessment tool of a criterion-referenced examination comprised of clinical cases tied to practice decisions, to examine the effect of varying audience response system (ARS) questions on student examination preparation, and to articulate guidelines for structuring examinations to maximize evaluation of student learning.Design. Multiple-choice items developed over 5 years were evaluated using Bloom’s Taxonomy classification, point biserial correlation, item difficulty, and grade distribution. In addition, examination items were classified into categories based on similarity to items used in ARS preparation.Assessment. As the number of items directly tied to clinical practice rose, Bloom’s Taxonomy level and item difficulty also rose. In examination years where Bloom’s levels were high but preparation was minimal, average grade distribution was lower compared with years in which student preparation was higher.Conclusion. Criterion-referenced examinations can benefit from systematic evaluation of their composition and effectiveness as assessment tools. Calculated design and delivery of classroom preparation is an asset in improving examination performance on rigorous, practice-relevant examinations.     

Pharmacy Students’ Ability to Think About Thinking

Objective. To investigate students’ metacognitive skills to distinguish what they know from what they do not know, to assess students’ prediction of performance on a summative examination, and to compare student-identified incorrect questions with actual examination performance in order to improve exam quality.Methods. Students completed a test-taking questionnaire identifying items perceived to be incorrect and rating their test-taking ability.Results. Higher performing students evidenced better metacognitive skills by more accurately identifying incorrect items on the exam. Most students (86%) underpredicted their performance on the summative examination (actual=73.6 ± 7.1 versus predicted=63.7 ± 10.5, p<0.05). Student responses helped refine items and resulted in examination changes.Conclusion. Metacognition is important to the development of life-long learning in pharmacy students. Students able to monitor what they know and what they do not know can improve their performance.     

Student Pharmacists’ Perceptions of Male Faculty Teaching Female-Specific Sex and Gender Health Topics

Objective. The primary objective of this study was to assess perceptions of second year pharmacy students regarding predominantly male faculty instructing them regarding female physiology, pathophysiology, and gender health topics.Methods. A cross-sectional survey was administered to second year pharmacy students at a small, private, non-profit college of pharmacy at the conclusion of their women’s health integrated pharmacotherapeutics course. Students were asked to rate their level of agreement with various statements using a five-point Likert scale on which 1=strongly disagree and 5=strongly agree.Results. Forty-seven students completed the survey (26 female and 21 male). The students indicated high agreement with statement: “Prior to the lectures, I felt comfortable having male faculty teach the majority of women''s health topics” (median=5, IQR=4-5). Students also indicated neutrality towards the statement: “The gender of the instructor is of importance in the didactic instruction of women’s health topics” (median=3, IQR=1-3). No significant differences were observed regarding student gender.Conclusion. Pharmacy students were generally amenable to male faculty teaching female-specific sex and gender health topics. This pattern was observed in both male and female students.     

Performance of Pharmacy Students in a Communications Course Delivered Online During the COVID-19 Pandemic

Objective. To examine pharmacy student readiness, reception, and performance in a communications course during the COVID-19 pandemic and to compare that with the performance of students who completed the same course in person the previous year.Methods. First-year Doctor of Pharmacy students (2020 cohort) enrolled in a professional communications course completed pre- and post-course surveys regarding their readiness for and changes in perception of online learning. Student learning was assessed using midterm and final examination grades. These grades were then compared with those of students who had completed the same course in person (on campus) the previous year (2019 cohort).Results. Students’ preference for face-to-face instruction decreased from the pre-course to the post-course survey as indicated by responses made using a five-point Likert-scale (difference in means = −1.59; p < .05). Their comfort level with online learning increased (difference in means = +0.38, p < .05) by the end of the course. Students did not perceive any appreciable changes in rapport with the instructor by the end of the study. Course performance of students in the online cohort did not differ significantly from that of the 2019 cohort (p>.05).Conclusion. This study demonstrated that first year PharmD students were already somewhat prepared for online learning when they began a communication course, with further adjustment occurring as the quarter progressed. Remote online learning did not seem to impact pharmacy student learning in this communications course conducted during the COVID-19 crisis.     

Anti-anxiety effect of a novel 5-HT3 receptor antagonist N-(benzo[d]thiazol-2-yl)-3-ethoxyquinoxalin-2-carboxamide (6k) using battery tests for anxiety in mice

Objective:

To investigate the anti-anxiety activity of “6k”, a novel 5-hydroxytryptamine type 3 (5-HT₃) receptor antagonist in in mice.

Materials and Methods:

Anti-anxiety activity of “6k” (1, 2, and 4 mg/kg, intraperitoneally (i.p.)) was evaluated in mice by behavioral tests such as elevated plus maze (EPM), open field test (OFT), light-dark box (L&D), and hole board test (HBT). Diazepam (2 mg/kg, i.p.) served as reference standard.

Results:

“6k” significantly (P < 0.05) increased the time and entries in open arm in EPM as compared to vehicle control group. Further, “6k” significantly (P < 0.05) increased the central and peripheral ambulation along with rearings and time in central area; whereas, reduced the fecal pellets in OFT as compared to vehicle control group. There was significant (P < 0.05) reduction in the latency to enter dark chamber; whereas, increased number of crossings and time in light chamber in L&D aversion test by treatment with “6k” as compared to vehicle control group. In HBT, “6k” significantly (P < 0.05) increased the number of head dipping and squares crossed; whereas, reduced the latency for first head dip and number of fecal pellets as compared to vehicle control group.

Conclusion:

A novel 5-HT₃ receptor antagonist has anti-anxiety action.KEY WORDS: 5-HT₃ antagonist, anxiety, elevated plus maze, open field test, hole board test     

Collaborative Examination Item Review Process in a Team-Taught,Self-Care Sequence

Objective. To improve examination item quality by educating and involving course instructors in evidence-based item review and encouraging use of this process in future courses.Methods. A peer-review process was implemented in a 2-course sequence (intervention) that involved training and review sessions before each examination and was compared to the previous year’s courses (control). Instructors completed a presurvey and postsurvey on training, experience, self-confidence, and self-rated success in multiple-choice item writing. Statistics were calculated for all items in the control and intervention sequences and compared using independent t tests. Items also were classified into levels based on difficulty and discrimination, and distribution into these levels was compared between sequences with independent t tests.Results. No significant difference was found between control and intervention sequence items with regard to mean difficulty (86.3% and 84.4%) or discrimination (0.23- and 0.25), respectively, although item classification distribution did appear to change between the control and intervention sequences’ subjective feelings of confidence, and success in item writing increased between presurvey and postsurvey. Confidence in ability to peer-review test items and to implement a formal item evaluation process also increased.Conclusion. Item statistics did not change significantly, but reviewed and edited items distributed more favorably into item statistic-based categories. This method of review positively affected instructors’ perceptions of their item-writing confidence and success and improved self-rated opinions of their ability to edit items and train others to do so.     

The Impact of Preceptor and Student Learning Styles on Experiential Performance Measures

Objectives. To identify preceptors’ and students’ learning styles to determine how these impact students’ performance on pharmacy practice experience assessments.Methods. Students and preceptors were asked to complete a validated Pharmacist’s Inventory of Learning Styles (PILS) questionnaire to identify dominant and secondary learning styles. The significance of “matched” and “unmatched” learning styles between students and preceptors was evaluated based on performance on both subjective and objective practice experience assessments.Results. Sixty-one percent of 67 preceptors and 57% of 72 students who participated reported “assimilator” as their dominant learning style. No differences were found between student and preceptor performance on evaluations, regardless of learning style match.Conclusion. Determination of learning styles may encourage preceptors to use teaching methods to challenge students during pharmacy practice experiences; however, this does not appear to impact student or preceptor performance.