An Assessment Model for Multidisciplinary, Team-Taught Integrated Pharmacy Courses

Objective. To design and implement an assessment model to effectively deliver integrated multidisciplinary team-taught pharmacy courses.Design. An assessment model was developed for an integrated pharmacotherapeutics course that focused on writing detailed learning objectives and matching them to examination questions. Qualitative assessment of learning objectives, course-embedded quantitative assessment, and objective assessments of examinations by subdiscipline were performed.Assessment. This model was assessed through course evaluations, faculty and course coordinator perceptions, and faculty and student focus groups, which provided data that facilitated effective integration and identified gaps and overlaps in content. The assessment of the examinations by discipline and the embedded quantitative assessment results identified previously unassessed and poorly performing objectives. Students believed the course contributed significantly to their professional growth and that it was one of the best-integrated courses, based in part on the improved teaching methods.Conclusions. A systematic assessment model that was developed for the effective delivery of multidisciplinary team-taught courses can be standardized and delivered despite changes in instructors for subsequent course offerings.     

A Progressive,Collaborative Process to Improve a Curriculum and Define an Assessment Program

Objective. To revise the University of Oklahoma College of Pharmacy’s professional program outcomes and create an assessment map using results from previous peer review and mapping of all professional courses and curricular streams of knowledge, skills, and attitudes (KSAs).Design. After consolidating 15 original program outcomes into 11 more precise outcome statements, defining KSAs for each, and getting faculty approval of them, the committee detailed measurable program expectations upon graduation for each outcome and created an assessment map identifying where KSAs were taught, how they were to be assessed, and the expected ability level (novice, competent, proficient) for each across the curriculum.Assessment. The committee’s work identified deficits, inconsistencies, and disproportionalities in professional program assessment. It recommended assessments to capture student achievement of each outcome, identified performance levels and criteria to measure outcomes progressively in each professional year, and outlined a process to provide students periodic reports on their progress in achieving each outcome.Conclusion. This work establishes a firm foundation for ongoing efforts to measure effectiveness of the professional program, especially in light of Accreditation Council for Pharmacy Education’s (ACPE) revised accreditation standards.     

Identifying the Gaps: Needs Assessment to Guide Development of a Dedicated Toxicology Curriculum for Emergency Medicine Residents

IntroductionThe American Board of Emergency Medicine identifies medical toxicology as an essential curricular element for emergency medicine (EM) residencies; however, access to medical toxicology education varies widely by institution. We hypothesized that EM residents are uncomfortable with core toxicology content and would be interested in  a dedicated toxicology curriculum.MethodsAn electronic needs assessment survey developed by experts in EM and medical toxicology was sent to residents and program leadership at nine EM programs participating in the Emergency Medicine Education Research Alliance (EMERA), a geographically diverse sampling of academic EM residency programs. We queried the presence of a current toxicology curriculum, interest in a dedicated toxicology curriculum, and comfort with core toxicology concepts for board examinations and in clinical practice. ResultsA total of 148 residents and 8 faculty leadership completed the survey.  Only 29% of resident respondents felt comfortable with toxicology concepts, and only 66% of respondents reported access to a toxicology curriculum. Of those without a known toxicology curriculum, most were interested in a formal curriculum. Faculty respondents reported 6/8 programs offered a toxicology curriculum. Faculty at the two programs without a formal curriculum expressed interest in a dedicated curriculum.Conclusions Emergency medicine residents remain uncomfortable with the core toxicology content in clinical practice. The majority of residents without a known toxicology curriculum would be interested in a dedicated toxicology curriculum.Supplementary InformationThe online version contains supplementary material available at 10.1007/s13181-021-00834-7.     

Development of a Reliable,Valid Annual Skills Mastery Assessment Examination

Objective

To develop a methodology for a reliable, valid annual skills mastery assessment examination to provide formative student feedback, inform curricular review, and comply with the Accreditation Council for Pharmacy Education (ACPE) Standards 2007.

Design

A sample of program-level ability-based outcomes skills were chosen for the examination. Test items were written, underwent quality control, and were scored for level of difficulty. Versions of the examination for first-, second-, third-, and fourth-year pharmacy students were developed and administered, the results were analyzed, reliability and validity were evaluated, and reports were generated. Item-writing guidelines, quality control procedures, and examination production steps were codified to create a criterion-referenced examination. Students and faculty advisors received detailed score reports and results were used to guide student performance and stimulate a review of curricular outcomes.

Assessment

Content, criterion, and construct validity were analyzed as defined in the literature for the intended use of this assessment tool. Data suggest the Annual Skills Mastery Assessment (ASMA) examination is both reliable and valid. Students and faculty members were surveyed regarding the usefulness of the examination. Results indicate general satisfaction with the assessment program.

Conclusion

A reasonably reliable, reasonably valid multiple-choice annual skills mastery assessment for selected outcomes statements providing formative feedback and informed curricular review was developed.     

Reliability of a Minimal Competency Score for an Annual Skills Mastery Assessment

Objective. To determine whether the modified Angoff process can be used to calculate a reliable minimal competency (“cut”) score for the Annual Skills Mastery Assessment (ASMA).Methods. Three panels of pharmacy faculty members used a modified Angoff method to create a minimal competency score for 60 previously used test items. The panels did not know which items were included. Data were analyzed to determine differences between rating sessions, faculty type, item difficulty, and rater scoring bias.Results. The cut score generated was not significantly different by session or faculty type. The range of cut scores varied by less than 3% per examination. Faculty panelists correctly predicted student performance on items grouped as easy, medium, and hard.Conclusion. A properly constructed faculty panel can determine a reliable cut score and accurately rank relative test item difficulty using the modified Angoff process.     

A Capstone Course with a Comprehensive and Integrated Review of the Pharmacy Curriculum and Student Assessment as a Preparation for Advanced Pharmacy Practice Experiences

Objective. To create a capstone course that provides a comprehensive and integrated review of the pharmacy curriculum with a broad range of assessment tools to evaluate student knowledge and skills as a final preparation prior to beginning fourth-year advanced pharmacy practice experiences (APPEs).Design. The capstone course was a 4 credit-hour, case-based course. Eight comprehensive cases were assigned to students over the course of the term. The cases were designed to mimic complex clinical scenarios that students were likely to encounter during an APPE. Students were required to prepare a written and oral presentation for each case and were assessed on material covered during the cases. Faculty members presented weekly reviews on selected topics such as calculations, pharmacokinetics, and pharmaceutical compounding. At the end of the course, students took an observed structured clinical examination (OSCE), which simulated the Georgia Board of Pharmacy Practical Examination, and a comprehensive examination designed to mimic the NAPLEX (North American Pharmacy Licensure Examination).Assessment. Evaluation of student outcomes was based on written and verbal presentations of the cases, multiple-choice examinations, a short-answer calculations examination, an “Errors and Omissions” examination, a standardized patient encounter, and pharmaceutical compounding examinations. Ninety-five percent of students successfully passed the course on their first attempt. Student feedback indicated satisfaction with the depth, breadth, and organization of material covered and felt that the course helped prepare them for APPEs.Conclusion. The culminating experience of the capstone course gave students a thorough review of practical, clinical, and communication skills and provided faculty members with feedback regarding the curriculum through robust assessment.     

Assessment of Pharmacy Student Professionalism Across a Curriculum

Objectives

To evaluate changes in professionalism across the curriculum among pharmacy students in different classes.

Methods

A professionalism instrument was administered early in the first (P1) year, upon completing the introductory pharmacy practice experiences (IPPE) near the end of the second (P2) year, and upon completing the advanced pharmacy practice experiences (APPE) at the end of the fourth (P4) year.

Results

The professionalism scale and its subscales were compared for the 3 time points for the class of 2009. Significant differences were noted in professionalism scores between the P1 and P4 years and for altruism, accountability, and honor/integrity subscale scores for the class of 2009. No significant differences were noted when the scores for 4 P1 classes, and 3 P2 classes were compared.

Conclusion

An increase in professionalism scores and altruism, accountability, and honor/integrity scores was demonstrated, providing evidence that the curricular and co-curricular activities in the school of pharmacy helped develop professionalism in the class of 2009 students.     

Professional Development Webinars for Pharmacists

Objectives. To evaluate pharmacists’ satisfaction with and reasons for enrolling in a series of continuing education webinars.Design. Webinars intended to provide timely and practical information for practicing pharmacists on a specific therapeutic area were created and presented monthly throughout 2011.Assessment. Survey responses of volunteers who completed at least 1 webinar were positive regarding the quality of the programming and the method of delivery. No significant differences were found in demographics or reasons for enrolling between participants who completed a single webinar and those who completed 2 or more.Conclusions. While the webinars received positive evaluations for quality, value, and relevance, the limited average number of webinars attended by each pharmacist was a concern, and several tactics will be implemented to address scheduling conflicts and other deterrents to repeat participation.     

Development and Assessment of the Multiple Mini-Interview in a School of Pharmacy Admissions Model

Objective. To describe the development, implementation, and evaluation of the multiple mini-interview (MMI) within a doctor of pharmacy (PharmD) admissions model.Methods. Demographic data and academic indicators were collected for all candidates who participated in Candidates’ Day (n=253), along with the score for each MMI station criteria (7 stations). A survey was administered to all candidates who completed the MMI, and another survey was administered to all interviewers to examine perceptions of the MMI.Results. Analyses suggest that MMI stations assessed different attributes as designed, with Cronbach alpha for each station ranging from 0.90 to 0.95. All correlations between MMI station scores and academic indicators were negligible. No significant differences in average station scores were found based on age, gender, or race.Conclusion. This study provides additional support for the use of the MMI as an admissions tool in pharmacy education.     

A Curriculum Review and Mapping Process Supported by an Electronic Database System

Curriculum mapping and review is now an expected continuous quality improvement initiative of pharmacy professional programs. Effectively implementing and sustaining this expectation can be a challenge to institutions of higher education and requires dedicated faculty members, a systematic approach, creativity, and—perhaps most importantly—demonstrated leadership at all levels of the institution. To address its specific situation and needs, the University of Oklahoma College of Pharmacy implemented a peer review process of ongoing curriculum mapping and evaluation. An electronic Pharmacy Curriculum Management System (PCMS) was developed to support faculty efforts to manage curricular data, monitor program outcomes, and improve communications to its stakeholders on 2 campuses and across the state.     

Implementation and Use of the Pharmacy Curriculum Outcomes Assessment at US Schools of Pharmacy

Objective. To describe how schools and colleges of pharmacy use the Pharmacy Curriculum Outcomes Assessment (PCOA) in relation to student assessment and curricular feedback.Methods. A survey was distributed to all programs that have implemented the PCOA. The survey was designed to assess 3 domains regarding the use of the PCOA: rationale for use, logistics of administration, and performance data review and distribution.Results. A 79% response rate (41/52) was obtained. The mix of responses was 93% current PCOA users and 7% past users. The most common reasons for PCOA use were for programmatic assessment and benchmarking. The examination was most frequently administered during the P3 year, with minimal stakes attached to performance. Significant differences in responses based on public vs private institution were seen with respect to length of accreditation of current PCOA users, messaging to students regarding performance, inclusion of results in student advising, and distribution of results to stakeholders.Conclusion. Programs were using the PCOA primarily as an assessment in the P3 year for reasons related to programmatic and curricular assessment. Some differences existed between public and private institutional PCOA use and examination-related processes and results distribution.     

Risk Assessment for Honeybees from Pesticide-Exposed Pollen

A method for assessing the risk for honeybees from pesticide exposure via pollen is proposed. Four pesticides, selected as markers, were monitored in pollen samples collected in two sampling areas, one located in an intensive agricultural area and the other far from direct pesticide impact. Analytical results were consistent with use patterns of the chemicals and their physico-chemical and persistence properties. For a preliminary estimate of bee exposure via pollen, both by ingestion and by contact, an exposure index was developed, based on physico-chemical properties, persistence and application rates. On the basis of the exposure estimates and acute toxicological data (ingestion and contact LD₅₀), Toxicity Exposure Ratios (TERs) were calculated as indicators of the risk for honeybees due to this particular exposure route. TER values were compared to Hazard Quotient (HQ), calculated as the ratio between application rate and the LC₅₀ value, according to European guidelines, showing a satisfactory agreement. The advantage of the above described procedures is that the environmental fate of the chemicals, and not only application rates, are taken into account. This approach may represent a preliminary tool for a comparative screening of the risk for pollinator insects due to this particular exposure route.     

Development of an Entrustment-Supervision Assessment Tool for Pharmacy Experiential Education Using Stakeholder Focus Groups

Objective. To devise a pharmacy-specific, expanded entrustable professional activities (EPA) entrustment-supervision scale that would frame preceptor ratings in a prospective and retrospective manner for use in experiential learning settings.Methods. A series of focus group sessions were conducted to solicit expert opinion on how to develop and refine two entrustment-supervision scales. Purposive sampling was used to identify experts from different professional groups (physicians, pharmacy experiential administrators, and pharmacy practice faculty) who had extensive knowledge regarding EPAs and at least one publication related to EPAs. Panelists were invited to participate via email. Three focus sessions were conducted via videoconferencing between June and September 2019. The primary outcome was development of a pharmacy-specific EPA entrustment-supervision assessment tool. Secondary outcomes were individual entrustment-supervision statements across five levels of the entrustment-supervision scale.Results. The focus group consisted of four pharmacy practice faculty, two experiential administrators, and one academic physician. Four concepts emerged from the focus group discussion: need for more granularity in entrustment-supervision scales; limitations due to differences in licensure requirements across the United States; present and ongoing use of expanded scales by schools and colleges of pharmacy; and uncertainty regarding how to rate entrustment-supervision when a student exhibits unprofessional behavior.Conclusion. A pharmacy-specific, expanded EPA entrustment-supervision scale will be useful to support longitudinal assessment of learners in experiential settings where an EPA framework is utilized. Determining when to use a prospective versus retrospective perspective requires further evaluation.     

皮肤毒理学检验中的替代方法和整合测试评估方法

近些年,随着实验动物3R原则的兴起,动物替代实验的研究越来越重要.为了使国产化妆品的开发和国际标准接轨,中国加快了替代实验方法在化妆品安全性检验中的应用.针对化妆品对皮肤可能造成的刺激性、致敏性和光毒性等不良反应,本文分别对化妆品皮肤毒理学检验中的替代实验方法及毒理学终点相应的整合测试评估方法(IATA)进行概...     

人血浆卡马西平浓度测定的LC-MS/MS方法开发、验证及在室间质评中的应用

摘 要 目的：建立并验证人血浆中卡马西平（CBZ）浓度测定的液质联用方法,并应用于室间质评样品的测定。方法: 使用含0.1%甲酸溶液和乙腈进行梯度洗脱,色谱柱：Kromasil C₁₈柱（50 mm×2.1 mm,5 μm）,流速：0.6 ml·min^-1,进样量：1 μl,柱温：40 ℃。在ESI⁺电离模式下,CBZ和内标噻氯匹定（IS）的监测离子对分别为m/z237.1>194.0和m/z264.1>125.0。结果: 卡马西平在5~1 000 ng·ml^-1范围内线性良好。低、中和高浓度的基质效应范围为93.23%~96.30%,回收率范围为101.27%~105.00%。结论:建立的测定CBZ血浆浓度的液质联用方法灵敏、准确、稳定,可用于CBZ的室间质评及药动学研究。     

我国健康状况评价指标的筛选

目的:探讨能反映我国健康状况有效评价指标。方法:采用德尔菲法、变异系数法、相关系数法、多元逐步回归分析法和主成分分析法对我国健康状况65项指标进行筛选。结果:经5种方法筛选出反映我国健康状况的评价指标共26项。结论:筛选出的26项指标组成我国健康状况评价指标体系,代表性好、独立性强、切实可行。     

Evaluation of an Instrument to Assess Students’ Personal and Professional Development During the Faculty Advising Process

Objective. To evaluate a tool designed to assess Doctor of Pharmacy (PharmD) students’ personal and professional development prior to beginning advanced pharmacy practice experiences (APPEs).Methods. A five-item instrument, entitled the Faculty Advisor’s Assessment of the Advisee (FAAA) tool, was developed to assess and monitor pharmacy students’ progress over the three-year didactic curriculum. Question anchors were created to describe characteristics exhibited by the student that matched categories of not engaged, beginning, emerging, or engaged. Possible FAAA composite scores ranged from 7 to 20. Using the FAAA tool, faculty advisors assessed their advisees’ values, engagement, self-awareness, professionalism, and leadership in 2017, 2018, and 2019. Individual and aggregate cohort reports were run and data for each of the three years were matched with students. To determine if the FAAA showed progression in assessed dimensions in the students during the first, second, and third professional (P1, P2, and P3) years, a Friedman test was performed. Cronbach alpha was used to assess the reliability of the instrument.Results. The data of 93 students were matched for the P1 through the P3 years. Median (IQR) for the FAAA composite score levels for the P1, P2, and P3 were 13 (12-16), 17 (15-19) and 18 (16-20), respectively. Significant differences existed at all timepoints compared, including from the P1 to P2, P2 to P3, and P1 to P3 years. The reliability of the FAAA scale was strong across all three years (winter 2017, α=0.87; winter 2018, α=0.89; and winter 2019, α=0.87). All items appeared worthy of retention as removal did not significantly increase their reliability.Conclusion. A five-item tool which assesses pharmacy students’ personal and professional development during the first three years of a PharmD program could be used by faculty advisors to assess student’s progress across the didactic curriculum.     

Finding and Using Readily Available Sources of Assessment Data

This paper provides the rationale for, examples of, and the collection and general uses of currently available, potentially underutilized academic program assessment data. Academic program assessment is essential for program improvement and accreditation, but commonly used assessment methods may not fully meet these needs. General assessment references, pharmacy education literature, and prior experiences were used to identify and discuss sources of potentially underutilized assessment data. Pre-course assessments, graded assignments, examinations, pharmacy experience evaluations, scoring rubrics, portfolios, progress testing, self-assessments, and classroom assessment techniques are potential sources of assessment data. Course evaluations and grades may also be useful. Selection should be based on need, availability, strength, and concerns. Challenges in data management may be best met through a centralized, integrated database. Careful selection of specific embedded and other assessments can be utilized to complete the development of a comprehensive, meaningful, and efficient program assessment plan.     

Addressing Competencies for the Future in the Professional Curriculum

This paper reviews the literature, analyzes current and future practice, develops a list of competencies necessary for future pharmacists, and provides recommendations to pharmacy''s academic enterprise regarding curricula of the future. Curricula of the future will center around 3 functional roles for pharmacists: patient-centered care, population-based care, and systems management; and must also foster the development of 5 cross-cutting abilities in student pharmacists: professionalism, self-directed learning, leadership and advocacy, interprofessional collaboration, and cultural competency. Future curricula must be developed in an evidence-based manner, focus less on information storage and retrieval, engage student pharmacists in a variety of highly interactive learning experiences, and expand experiential learning opportunities throughout all years.     

用综合评价方法构建医院药学人员绩效考核体系

目的:探索构建医院药学人员绩效考核体系。方法:通过德尔菲法咨询院内、外9位专家,确定绩效考核体系构建方法,即采用综合评价方法,包括平衡计分卡法、关键绩效指标法和360度考评法等方法,形成初步的考核指标及权重,再通过问卷方式调查北京市9家三级甲等综合医院360名药学工作人员(占药学工作人员总数的37%),确定最终的考核体系。结果:根据医院药学不同岗位的职责,确定了各岗位的绩效考核指标及权重,指标包括日常工作、顾客评价、内部流程和学习与成长4个维度,赋予日常工作指标总分100分,后3个维度的指标采用在第1个指标得分基础上加减分的方式,以分值占总分的比重来体现指标的权重,从而构建了一套绩效考核体系。结论:通过综合评价方法构建的医院药学人员绩效考核体系较为全面、客观,为实施有效的绩效管理打下基础。