Implementation and Use of the Pharmacy Curriculum Outcomes Assessment at US Schools of Pharmacy

Objective. To describe how schools and colleges of pharmacy use the Pharmacy Curriculum Outcomes Assessment (PCOA) in relation to student assessment and curricular feedback.Methods. A survey was distributed to all programs that have implemented the PCOA. The survey was designed to assess 3 domains regarding the use of the PCOA: rationale for use, logistics of administration, and performance data review and distribution.Results. A 79% response rate (41/52) was obtained. The mix of responses was 93% current PCOA users and 7% past users. The most common reasons for PCOA use were for programmatic assessment and benchmarking. The examination was most frequently administered during the P3 year, with minimal stakes attached to performance. Significant differences in responses based on public vs private institution were seen with respect to length of accreditation of current PCOA users, messaging to students regarding performance, inclusion of results in student advising, and distribution of results to stakeholders.Conclusion. Programs were using the PCOA primarily as an assessment in the P3 year for reasons related to programmatic and curricular assessment. Some differences existed between public and private institutional PCOA use and examination-related processes and results distribution.     

A Scoping Review of the Pharmacy Curriculum Outcomes Assessment Literature

Objective. To identify themes, gaps, and sources of evidence from the literature regarding the Pharmacy Curriculum Outcomes Assessment (PCOA) to inform practice and additional areas for research in pharmacy education.Findings. Nineteen articles describing the administration and use of PCOA were identified. Since PCOA was made a curricular requirement by the Accreditation Council of Pharmacy Education in 2016, the focus of literature related to the PCOA has shifted from administration practices (four articles published before 2016 vs two articles published since) to determining models that may predict student performance on the assessment (two vs five articles) or how the examination might be used to predict future performance (one vs seven articles), especially on the North American Pharmacist Licensing Examination. While there is a growing body of literature focused on the PCOA’s utility for measuring performance, few variables have been consistently used.Summary. This review found no studies with objectives that aligned with the initial intended use of the PCOA as defined by the National Association of Boards of Pharmacy, which included tracking individual student performance throughout the curriculum, benchmarking programs against other programs, and evaluating whether a program was meeting their desired outcomes. Additionally, no consensus across the Academy was found as to the proper use of the PCOA, and a paucity of literature exists regarding how the PCOA informs schools and colleges about the effectiveness of their curriculum. There is a need for the Academy to establish a uniform application for the PCOA in pharmacy schools, assess the resources that programs need to administer this required assessment, and determine the utility of the PCOA to measure curricular effectiveness and/or student performance.     

Academic Progression and Retention Policies of Colleges and Schools of Pharmacy

Objective. To describe academic progression and retention policies used by US colleges and schools of pharmacy.Methods. Student handbooks on the Web sites of 122 colleges and schools of pharmacy were reviewed between February 2012 and May 2012.Results. Data were available and obtained from 98 (80%) programs. Most used grade point average (GPA) as a criterion for progression, with 66% requiring a minimum GPA of 2.0. Cumulative GPA was the most frequently used criteria for probation. Most handbooks did not address remediation, but 38% noted that a failed course could only be retaken once. The most common criteria for dismissal were the cumulative number of times a student was on probation. The graduation requirements of most programs were a cumulative GPA of 2.0 and completion of the program within 6 years of enrollment. Conclusions. Colleges and schools of pharmacy use various criteria for academic progression and retention and frequently provide incomplete or inadequate information related to probation, progression, and dismissal. Information regarding remediation and academic performance during experiential learning is lacking. A clearinghouse containing institutional data related to progression and retention would assist programs in developing academic policies. The study also highlights the need for ACPE to ensure this information is provided to students.     

Systematic Review of Predictors of Success for the North American Pharmacist Licensure Examination

Objective. Studies have examined possible predictors of success on the North American Pharmacist Licensure Examination (NAPLEX). This systematic review investigated the literature on potential predictors of success on the NAPLEX.Findings. Articles were included in the review if they studied student characteristics and academic performance as independent variables and NAPLEX scores and/or pass rates as an outcome. Data were extracted from each article for students’ demographics or variables, sample size, methods of statistical analyses, and results reporting correlation or predictability. From 40 articles retrieved from the initial search and sorting, 20 studies were included in the final review per inclusion criteria. Three studies included all the pharmacy programs, 15 were single-institution studies, two were multi-institution studies, and four had been published as posters. Among 30 different variables identified as potential predictors of success on the NAPLEX, the most examined variables were student age at matriculation, having a prior degree, Pharmacy College Admission Test (PCAT) scores, cumulative pharmacy school grade point average (GPA), overall Pharmacy Curriculum Outcomes Assessment (PCOA) scores, and PCOA content areas scores. Positively correlated factors included PCAT scores, not having a prior degree, prepharmacy and pharmacy school GPA, institutional characteristics, and PCOA scores. Negatively correlated factors included older age at admission.Summary. Cumulative pharmacy school GPA and PCOA scores were predictors of NAPLEX success consistently in the studies. The effects of preadmission student characteristics on NAPLEX success varied and were not consistently correlated or predictive.     

Assessment of Full-time Faculty Preceptors By Colleges and Schools of Pharmacy in the United States and Puerto Rico

Objective. To identify the manner in which colleges and schools of pharmacy in the United States and Puerto Rico assess full-time faculty preceptors. Methods. Directors of pharmacy practice (or equivalent title) were invited to complete an online, self-administered questionnaire.Results. Seventy of the 75 respondents (93.3%) confirmed that their college or school assessed full-time pharmacy faculty members based on activities related to precepting students at a practice site. The most commonly reported assessment components were summative student evaluations (98.5%), type of professional service provided (92.3%), scholarly accomplishments (86.2%), and community service (72.3%). Approximately 42% of respondents indicated that a letter of evaluation provided by a site-based supervisor was included in their assessment process. Some colleges and schools also conducted onsite assessment of faculty members.Conclusions. Most colleges and schools of pharmacy assess full-time faculty-member preceptors via summative student assessments, although other strategies are used. Given the important role of preceptors in ensuring students are prepared for pharmacy practice, colleges and schools of pharmacy should review their assessment strategies for full-time faculty preceptors, keeping in mind the methodologies used by other institutions.     

The Evolving Role of Historically Black Pharmacy Schools in a Changing Environment

Objective. To provide a comprehensive review of the contributions of historically Black colleges and universities (HBCUs) to creating a diverse pharmacist workforce and identify opportunities for future contributions. This was accomplished by comparing enrollment at HBCUs to overall US enrollment of African American pharmacy students and then comparing those numbers to national enrollment of pharmacy students, analyzing contributions of HBCUs to underrepresented pharmacy student enrollment, evaluating overall changes to pharmacy school enrollment and impact on enrollment at HBCUs, and identifying areas of opportunity to enhance the future contributions of HBCUs.Findings. There are six HBCU pharmacy schools in the United States. Although HBCU pharmacy schools made up only 4% of the total number of US pharmacy schools, they accounted for an average of 22.8% of the total African American student enrollment in pharmacy schools over a five-year period (2015-2019). An average of 13.8% of the total population of underrepresented people of color (UPOC) enrolled in US pharmacy schools from 2015-2019 attended an HBCU pharmacy school.Summary. Historically Black colleges and universities have consistently made significant contributions to the total pharmacy school population of African Americans, as well as that of Native Hawaiian and other Pacific Islanders and American Indian and Alaska Natives. These institutions have an opportunity to enhance their impact and serve in graduating the diverse pharmacy workforce needed in the future. Historically Black colleges and universities can achieve this goal by making significant efforts to recruit LatinX pharmacy students and by increasing their recruitment of African Americans, Native Hawaiian and other Pacific Islanders, and American Indian and Alaska Natives.     

An assessment of the level of moral development of American and Canadian pharmacy students

Context — Moral reasoning assessment is an assessment of conceptual adequacy of moral thinking. It attempts to tap the basic conceptual frameworks that individuals use to analyse social‐moral problems and to judge the proper course of action. It is a significant construct because of its pragmatic and positive relationship to clinical performance in health professionals. Thus, pharmacy students at higher levels of moral reasoning can be expected to perform at higher levels on clinical performance measurements. Objective — To examine the moral reasoning of pharmacy students at schools of pharmacy in the United States and Canada. Method — Rest's Defining Issues Test (DIT) was used as a surrogate measure of pharmacy students' moral reasoning and was completed by all first‐year pharmacy students at one school of pharmacy in the US and one in Canada. Results — The results indicated that significant differences existed between the Canadian and American pharmacy students. The Canadian students scored significantly higher than their American counterparts. The mean moral reasoning score of both classes was lower than previous empirical research results from other first‐year health professional students. The results are discussed along with potential educational interventions and suggestions for selecting pharmacy students to schools of pharmacy. Conclusion — This is the first study to have assessed moral reasoning in pharmacy students. The reasons for differences between US and Canadian students are unknown. The lower scores of pharmacy students compared with other student health professionals indicate that further research is needed.     

An Automated Competency-based Student Performance Assessment Program for Advanced Pharmacy Practice Experiential Programs

Objectives

To describe the development and preliminary outcomes of the System of Universal Clinical Competency Evaluation in the Sunshine State (SUCCESS) for preceptors to assess students'' clinical performance in advanced pharmacy practice experiences (APPEs).

Design

An Internet-based APPE assessment tool was developed by faculty members from colleges of pharmacy in Florida and implemented.

Assessment

Numeric scores and grades derived from the SUCCESS algorithm were similar to preceptors'' comparison grades. The average SUCCESS GPA was slightly higher compared to preceptors'' scores (0.02 grade points).

Conclusions

The SUCCESS program met its goals, including establishing a common set of forms, standardized assessment criteria, an objective document that is accessible on the Internet, and standardized grading, and reducing pressure on preceptors from students concerning their grades.     

Death education: a survey of U.S. colleges and schools of pharmacy.

This survey was conducted to assess the present state of death education available to pharmacy students at both the BS and PharmD degree levels. Eighty-five percent of the colleges and schools of pharmacy located in the United States provided information concerning: (i) attitudes toward teaching death education; (ii) present offerings; (iii) academic background of instructors and departments responsible for death education programs; and (iv) course information. It was determined that 38 colleges of pharmacy offered some form of death education in their curricula during 1985. However, only 12 schools offered a full semester course, and many schools offered death education as an elective through a discipline other than pharmacy. As a result, a majority of graduates are still leaving pharmacy schools without any instruction in death education. The implications of these and other findings are discussed.     

Interprofessional education in introductory pharmacy practice experiences at US colleges and schools of pharmacy

Objective. To assess the extent to which US colleges and schools of pharmacy are incorporating interprofessional education into their introductory pharmacy practice experiences (IPPEs), and to identify barriers to implementation; characterize the format, structure, and assessment; and identify factors associated with incorporating interprofessional education in IPPEs.Methods. An electronic survey of 116 US colleges and schools of pharmacy was conducted from March 2011 through May 2011.Results. Interprofessional education is a stated curricular goal in 78% of colleges and schools and consistently occurred in IPPEs in 55%. Most colleges and schools that included interprofessional education in IPPEs (70%) used subjective measures to assess competencies, while 17.5% used standardized outcomes assessment instruments. Barriers cited by respondents from colleges and schools that had not implemented interprofessional education in IPPEs included a lack of access to sufficient healthcare facilities with interprofessional education opportunities (57%) and a lack of required personnel resources (52%).Conclusions. Many US colleges and schools of pharmacy have incorporated interprofessional education into their IPPEs, but there is a need for further expansion of interprofessional education and better assessment related to achievement of interprofessional education competencies in IPPEs.     

Self-Efficacy and Self-Esteem in Third-Year Pharmacy Students

Objective. To identify the experiential and demographic factors affecting the self-efficacy and self-esteem of third-year pharmacy (P3) students.Methods. A 25-item survey that included the Rosenberg Self-Esteem Scale and the General Self-Efficacy Scale, as well as types and length of pharmacy practice experiences and demographic information was administered to doctor of pharmacy (PharmD) students from 5 schools of pharmacy in New England at the completion of their P3 year.Results. The survey response rate was approximately 50% of the total target population (399/820). Students with a grade point average (GPA)≥3.0 demonstrated a higher significant effect from unpaid introductory pharmacy practice experiences (IPPEs) on their self-efficacy scores (p<0.05) compared to students with lower GPAs. Students who had completed more than the required amount of pharmacy experiences had higher levels of self-efficacy and self-esteem (p<0.05). Ethnicity also was related to students’ levels of self-efficacy and self-esteem.Conclusion. Self-efficacy and self-esteem are two important factors in pharmacy practice. Colleges and schools of pharmacy should ensure that students complete enough practice experiences, beyond the minimum of 300 IPPE hours, as one way to improve their self-efficacy and self-esteem.     

Pharmacy students' test-taking motivation-effort on a low-stakes standardized test

Objective

To measure third-year pharmacy students'' level of motivation while completing the Pharmacy Curriculum Outcomes Assessment (PCOA) administered as a low-stakes test to better understand use of the PCOA as a measure of student content knowledge.

Methods

Student motivation was manipulated through an incentive (ie, personal letter from the dean) and a process of statistical motivation filtering. Data were analyzed to determine any differences between the experimental and control groups in PCOA test performance, motivation to perform well, and test performance after filtering for low motivation-effort.

Results

Incentivizing students diminished the need for filtering PCOA scores for low effort. Where filtering was used, performance scores improved, providing a more realistic measure of aggregate student performance.

Conclusions

To ensure that PCOA scores are an accurate reflection of student knowledge, incentivizing and/or filtering for low motivation-effort among pharmacy students should be considered fundamental best practice when the PCOA is administered as a low-stakes testKey words: Pharmacy Curriculum Outcomes Assessment (PCOA), testing, motivation, examination     

Teaching medication adherence in US colleges and schools of pharmacy

Objective. To determine and describe the nature and extent of medication adherence education in US colleges and schools of pharmacy.Methods. A mixed-methods research study was conducted that included a national survey of pharmacy faculty members, a national survey of pharmacy students, and phone interviews of 3 faculty members and 6 preceptors.Results. The majority of faculty members and students agreed that background concepts in medication adherence are well covered in pharmacy curricula. Approximately 40% to 65% of the students sampled were not familiar with several adherence interventions. The 6 preceptors who were interviewed felt they were not well-informed on adherence interventions, unclear on what students knew about adherence, and challenged to provide adherence-related activities for students during practice experiences because of practice time constraints.Conclusions. Intermediate and advanced concepts in medication adherence, such as conducting interventions, are not adequately covered in pharmacy curriculums; therefore stakeholders in pharmacy education must develop national standards and tools to ensure consistent and adequate medication adherence education.     

Attributes of Colleges and Schools of Pharmacy in the United States

Objectives

To compare the attributes of US colleges and schools of pharmacy and describe the extent of change to the pharmacy education enterprise associated with the addition of new schools.

Methods

Attributes analyzed included whether the college or school of pharmacy was old or new, public or private, secular or faith-based, and on or not on an academic health center (AHC) campus; had 3- or 4- year programs; and had PhD students enrolled. PharmD student enrollment-to-faculty ratios and junior-to-senior faculty ratios also were examined.

Results

Of the new colleges/schools, 76% were private and 79% were not located on a campus with an AHC; 6% had PhD enrollment compared with 80% of old colleges/schools. Faculty ratios were related to several college/school attributes, including the presence or absence of PhD students and whether the college/school was public or private.

Conclusions

Attributes of new colleges and schools of pharmacy have changed the overall profile of all colleges and schools of pharmacy. For example, smaller percentages of all colleges and schools of pharmacy are public and have PhD enrollees.     

Multisite Study Assessing the Effect of Cognitive Test Anxiety on Academic and Standardized Test Performance

Objective. To evaluate the degree of cognitive test anxiety (CTA) present in student pharmacists at multiple pharmacy programs in the United States and to determine if there are associations between self-reported CTA and relevant academic outcomes.Methods. All 2018-2019 advanced pharmacy practice experience (APPE) students from three US Doctor of Pharmacy (PharmD) programs (N=260) were invited to participate in the study. Participants completed a validated 37-question survey that included the Cognitive Test Anxiety Scale-2 (CTAS-2) along with demographics-related questions. Responses were analyzed using analysis of variance (ANOVA), Kruskal Wallace, and multiple linear regression where appropriate.Results. One hundred twenty-four students (48%) from the three programs participated in the study, and the individual data of 119 (46%) were included in the final analysis. Twenty-two students (18.5%) were classified as having high CTA, 41 (34.5%) as having moderate CTA, and 56 (47.1%) as having low CTA. High CTA predicted a 8.9 point lower NAPLEX total scaled score after accounting for other variables and was also correlated with lower cumulative didactic GPA, performance on the Pharmacy Curriculum Outcomes Assessment (PCOA), and increased likelihood of requiring course remediation.Conclusion. High cognitive test anxiety affects 18% of pharmacy students and may significantly impact their performance on a variety of traditional student success measures, including the NAPLEX. Pharmacy educators should consider further use and adoption of test anxiety measurements to identify and assist potentially struggling students.     

Instrument to measure psychological contract violation in pharmacy students

Objectives

To adapt and evaluate an instrument that measures perceived psychological contract violations in pharmacy students by schools and colleges of pharmacy.

Design

A psychological contract violations measure was developed from existing literature and the 1997 ACPE Guidelines and pilot-tested with second-year pharmacy students at 2 schools of pharmacy. A revised measure then was administered to second-year pharmacy students at 6 schools of pharmacy. Using a 5-point Likert-type scale, participants were asked to indicate the level of obligations they received compared to what was promised by the school of pharmacy.

Results

Exploratory factor analysis on the psychological contract violations measure was conducted using principal components analysis resulting in 7 factors, which led to a revised measure with 26 items. Using a sample of 339 students, the proposed 7-factor measurement model was tested using confirmatory factor analysis. In general, the results supported the hypothesized model. The final 23-item scale demonstrated both reliability and validity. Some students perceived certain aspects of the psychological contract that exists with their school of pharmacy were being violated.

Conclusion

The psychological contract violations measure may serve as a valuable tool in helping to identify areas where their students believe that schools/colleges of pharmacy have not fulfilled promised obligations.     

Teaching Evaluation Practices in Colleges and Schools of Pharmacy

Objective

To document teaching evaluation practices in colleges and schools of pharmacy.

Methods

A 51-item questionnaire was developed based on the instrument used in a previous study with modifications made to address changes in pharmacy education. An online survey service was used to distribute the electronic questionnaire to the deans of 98 colleges and schools of pharmacy in the United States.

Results

Completed surveys were received from 89 colleges and schools of pharmacy. All colleges/schools administered student evaluations of classroom and experiential teaching. Faculty peer evaluation of classroom teaching was used by 66% of colleges/schools. Use of other evaluation methods had increased over the previous decade, including use of formalized self-appraisal of teaching, review of teaching portfolios, interviews with samples of students, and review by teaching experts. While the majority (55%) of colleges/schools administered classroom teaching evaluations at or near the conclusion of a course, 38% administered them at the midpoint and/or conclusion of a faculty member''s teaching within a team-taught course. Completion of an online evaluation form was the most common method used for evaluation of classroom (54%) and experiential teaching (72%).

Conclusion

Teaching evaluation methods used in colleges and schools of pharmacy expanded from 1996 to 2007 to include more evaluation of experiential teaching, review by peers, formalized self-appraisal of teaching, review of teaching portfolios, interviews with samples of students, review by teaching experts, and evaluation by alumni. Procedures for conducting student evaluations of teaching have adapted to address changes in curriculum delivery and technology.     

Addressing the Need for Ethnic and Racial Diversity in the Pipeline for Pharmacy Faculty

Objective. To analyze the ethnic and racial diversity of faculty in pharmacy, medicine, and dentistry in the United States and suggest how the pipeline for pharmacy academe can be diversified.Methods. A retrospective analysis of the representativeness of faculty at schools and colleges of pharmacy was compared to that in schools and colleges of medicine and dentistry. The range of ethnic and racial diversity across top schools of pharmacy, historically black colleges and universities (HBCUs), and newer schools of pharmacy was evaluated for both faculty and students for the year 2019-2020. The ethnic and racial diversity in residency and fellowship programs along with graduation rates provided insight into the available pipeline for future pharmacy faculty.Results. Faculty in pharmacy, medicine, and dentistry demonstrated similarly low representation of underrepresented minorities (URMs) compared to their composition within the US population. Dentistry had the largest percentage of URMs (13.9%), compared with 8.5% in pharmacy and 7.1% in medicine. Five HBCUs contributed 32.8% of all Black faculty, yet their graduates had comparatively low residency match rates. The ratio of URM students to non-URM students in post-PharmD and graduate training programs is lower than the ratio of URM students to non-URM students in pharmacy programs.Conclusion. Lack of access to postgraduate residency or fellowship training programs is a major barrier to progression to pharmacy academe and impacts URMs more significantly. Barriers to advanced training must be removed or decreased to create the needed diverse faculty candidates for pharmacy academe. Without intervention, students in pharmacy programs will be primarily trained by non-URM faculty, which may impact how graduates provide care in an increasingly diverse patient population.