Effectiveness of a rural longitudinal integrated clerkship in preparing medical students for internship

Background: We interviewed graduates from the first two cohorts of a postgraduate medical program that had a senior year longitudinal integrated clerkship (LIC) in a practice setting in rural New South Wales, Australia to determine how well their training prepared them to be junior doctors (3–4 years after graduation), and what aspects of that training they thought were particularly useful.

Methods: In-depth interviews.

Results: Fourteen junior doctors were interviewed. Participants reported feeling well prepared in ability to develop close relationships with clinical supervisors, good clinical and procedural skills, ability to work autonomously and work in teams, knowledge of health systems, ability to ensure self-care, and professionalism. Consensus view was that a rural placement was an excellent way to learn medicine for a variety of reasons including relationships with clinicians, less competition for access to patients, and opportunities to extend their clinical skills and act up to intern level.

Conclusion: The advantages we found in the training these junior doctors received which prepared them well for internship were integral both to the longitudinal, unstructured placement, and to the fact that it was carried out in a rural area. The two aspects of these placements appear to act synergistically, reinforcing the learning experience.     

Uncover it,students would learn leadership from Team-Based Learning (TBL): The effect of guided reflection and feedback

Context: Little is known about best practices for teaching and learning leadership through Team-Based learning? (TBL?) with medical students. We hypothesized that guided reflection and feedback would improve shared leadership and shared leadership capacity, and enhance team decision quality in TBL teams. We used the Kolb experiential learning theory as the theoretical framework.

Method: The study was conducted at Tehran University of Medical Sciences. Three TBL sessions with 206 students (39 teams) participated in the study. Using a quasi-experimental design, one batch received guided reflection and feedback on their team leadership processes (n?=?20 teams) and the other received only TBL (n?=?19 teams). Observers measured shared leadership using a checklist. Shared leadership capacity was measured using a questionnaire. Scores on a team application exercise were used to assess quality of team decisions.

Results: Evidence did not support our first hypothesis that reflection and feedback enhance shared leadership in TBL teams. Percentages of teams displaying shared leadership did not differ between intervention and control groups in sessions 1 (p?=?0.6), 2 (p?=?1) or 3 (p?=?1). The results did not support the second hypothesis. We found no difference in quality of decision making between the intervention and control groups for sessions 1 (p?=?0.77), 2 (p?=?0.23), or 3 (p?=?0.07). The third hypothesis that the reflection and feedback would have an effect on shared leadership capacity was supported (T?=??8.55, p?>?0.001 adjusted on baseline; T?=??8.55, p?>?0.001 adjusted on gender).

Discussion and conclusion: We found that reflection and feedback improved shared leadership capacity but not shared leadership behaviors or team decision quality. We propose medical educators who apply TBL, should provide guided exercise in reflection and feedback so that students may better understand the benefits of working in teams as preparation for their future roles as leaders and members of health care teams.     

Evolutions in clinical reasoning assessment: The Evolving Script Concordance Test

Introduction: Script concordance testing (SCT) is a method of assessment of clinical reasoning. We developed a new type of SCT case design, the evolving SCT (E-SCT), whereby the patient’s clinical story is “evolving” and with thoughtful integration of new information at each stage, decisions related to clinical decision-making become increasingly clear.

Objectives: We aimed to: (1) determine whether an E-SCT could differentiate clinical reasoning ability among junior residents (JR), senior residents (SR), and pediatricians, (2) evaluate the reliability of an E-SCT, and (3) obtain qualitative feedback from participants to help inform the potential acceptability of the E-SCT.

Methods: A 12-case E-SCT, embedded within a 24-case pediatric SCT (PaedSCT), was administered to 91 pediatric residents (JR: n?=?50; SR: n?=?41). A total of 21 pediatricians served on the panel of experts (POE). A one-way analysis of variance (ANOVA) was conducted across the levels of experience. Participants’ feedback on the E-SCT was obtained with a post-test survey and analyzed using two methods: percentage preference and thematic analysis.

Results: Statistical differences existed across levels of training: F?=?19.31 (df?=?2); p?p?p?p?α) was 0.83. Participants found the E-SCT engaging, easy to follow and true to the daily clinical decision-making process.

Conclusions: The E-SCT demonstrated very good reliability and was effective in distinguishing clinical reasoning ability across three levels of experience. Participants found the E-SCT engaging and representative of real-life clinical reasoning and decision-making processes. We suggest that further refinement and utilization of the evolving style case will enhance SCT as a robust, engaging, and relevant method for the assessment of clinical reasoning.     

“But Dr Google said…” – Training medical students how to communicate with E-patients

Abstract

Purpose: Patients who have access to information online may feel empowered and also confront their physicians with more detailed questions. Medical students are not well-prepared for dealing with so-called “e-patients.” We created a teaching module to deal with this, and evaluate its effectiveness.

Method: Senior medical students had to manage encounters with standardized patients (SPE) in a cross-over design. They received blended-learning teaching on e-patients and a control intervention according to their randomization group (EI/LI?=?early/late intervention). Each SPE was rated by two blinded video raters, the SP and the student.

Results: N?=?46 students could be included. After the intervention, each group (EI, LI) significantly improved their competency in dealing with e-patients as judged by expert video raters (EI: M_T0?=?9.75 (2.51) versus M_T1?=?16.60 (2.80); LI: M_T0?=?8.70 (2.14) versus M_T2?=?15.20 (2.84); both p < 0.001) and SP (EI: M_T0?=?24.13 (4.83) versus M_T1?=?26.52 (3.06); LI: M_T0?=?23.37 (3.10) versus M_T2?=?27.47 (4.38); both p < 0.001). Students’ rating showed a similar non-significant trend.

Conclusions: Students, SP and expert video raters determined that blended-learning teaching can improve students’ competencies when dealing with e-patients. Within the study period, this effect was lasting; however, further studies should look at long-term outcomes.     

The OSCE progress test – Measuring clinical skill development over residency training

Purpose: The purpose of this study was to explore the use of an objective structured clinical examination for Internal Medicine residents (IM-OSCE) as a progress test for clinical skills.

Methods: Data from eight administrations of an IM-OSCE were analyzed retrospectively. Data were scaled to a mean of 500 and standard deviation (SD) of 100. A time-based comparison, treating post-graduate year (PGY) as a repeated-measures factor, was used to determine how residents’ performance progressed over time.

Results: Residents’ total IM-OSCE scores (n?=?244) increased over training from a mean of 445 (SD?=?84) in PGY-1 to 534 (SD?=?71) in PGY-3 (p?<?0.001). In an analysis of sub-scores, including only those who participated in the IM OSCE for all three years of training (n?=?46), mean structured oral scores increased from 464 (SD?=?92) to 533 (SD?=?83) (p?<?0.001), physical examination scores increased from 464 (SD?=?82) to 520 (SD?=?75) (p?<?0.001), and procedural skills increased from 495 (SD?=?99) to 555 (SD?=?67) (p?=?0.033). There was no significant change in communication scores (p?=?0.97).

Conclusions: The IM-OSCE can be used to demonstrate progression of clinical skills throughout residency training. Although most of the clinical skills assessed improved as residents progressed through their training, communication skills did not appear to change.     

Undergraduate medical students’ perspectives of skills,uses and preferences of information technology in medical education: A cross-sectional study in a Saudi Medical College

Abstract

Background: Information technology (IT) is widely used in medical education. However, there are not enough studies about IT uses and preferences among traditional and problem-based learning (PBL) medical students.

Aim: To compare IT skills, uses and preferences for education between traditional and PBL medical students’.

Method: A cross-sectional study; a modified Educause Center for Analysis and Research online survey was sent to traditional curriculum 5th and PBL 4th year medical students of King Saud University.

Results: Most of the responding 176 students prefer mobile devices and moderate amount of IT in education. Fourth and fifth year students perceived high academic value of Google (94.2 vs. 86.7%, p?=?0.34), YouTube (90.7 vs. 92.2%, p?=?0.83) and PubMed (83.7 vs. 86.7%, p?=?0.06). More 4th year than 5th year students rated themselves as skilled in learning management system (54.7 vs. 21.1%, p?=?0.0001) and Smartboard use (40.7 vs. 23.3%, p?=?0.04). Most students rated faculty IT skills as effective. Students agreed that technology helps working faster (95.5%) and make learning creative (85.9%).

Conclusions: More integration of information literacy and IT training in medical curricula is needed to enhance better utilization of full features of IT resources available for learning and problem solving. National multi-institutional studies are recommended.     

A unique flipped classroom approach shows promising results in physician assistant education

Abstract

Aims: The flipped classroom (FC) model utilizes pre-class foundational learning with in-class higher-order application exercises. FC approaches have reported positive perceptions and mixed academic performance outcomes in medical education; however, little evidence exists on their impact in the 56-week didactic curriculum of Physician Assistant (PA) education.

Methods: A 4-week FC curriculum was piloted in a PA behavioral medicine course (n?=?34). Before class, students completed online video modules. During class, students completed a pre-class quiz and answered clinical case-based questions in a think-pair-share format.

Results: Students rated in-class time higher than a lecture-based segment of the same course (p?=?0.028). Assessment of achievement emotions demonstrated increased enjoyment (p?=?0.028) and decreased boredom (p?<?0.001) in the FC curriculum with no difference in anxiety. The FC curriculum produced a 57.2?min increase in pre-class preparation time with no change in post-class studying time. Compared to historical lecture-based controls (n?=?35), students in the FC curriculum scored significantly higher on clinical vignette questions (p?=?0.019) with equivalent performance on other question formats.

Conclusions: This study offers the first positive outcomes of an FC approach in PA education and provides a framework for improving academic performance while scaling back in-class time.     

Effects of Team-Based Learning on short-term and long-term retention of factual knowledge

Purpose: We investigated the effect of Team-Based Learning (TBL) on long-term retention of knowledge in comparison to a traditional curriculum.

Methods: As TBL was incorporated into our curriculum in the 2008–2009 academic year, students were compared with those who received the traditional curriculum the year prior. Students in both the groups completed multiple-choice knowledge test at four time points spanning two years. Test performance was compared at each time point to assess changes in knowledge retention as a function of time.

Results: Baseline knowledge did not differ significantly between the TBL and control groups [51% versus 46%; t(84)?=?0.91, p?=?0.37, d?=?0.20]. Performance improved after the course for both the groups, but was significantly higher in the TBL group [79% versus 59%; t(84)?=?4.96, p?=?0.000004, d?=?0.95]. However, when assessed prior to the pediatrics clerkship, learning gains from TBL had largely disappeared and the small difference in performance was not significant [57% versus 51%; t(84)?=?1.51, p?=?0.14, d?=?0.32].

Conclusion: Incorporating TBL into the pre-clinical pediatrics curriculum led to large gains in knowledge over the short-term, but these gains did not persist. Further research should focus on extending the impact of TBL on long-term knowledge retention.     

A feedback system in residency to evaluate CanMEDS roles and provide high-quality feedback: Exploring its application

Introduction: Residents benefit from regular, high quality feedback on all CanMEDS roles during their training. However, feedback mostly concerns Medical Expert, leaving the other roles behind. A feedback system was developed to guide supervisors in providing feedback on CanMEDS roles. We analyzed whether feedback was provided on the intended roles and explored differences in quality of written feedback.

Methods: In the feedback system, CanMEDS roles were assigned to five authentic situations: Patient Encounter, Morning Report, On-call, CAT, and Oral Presentation. Quality of feedback was operationalized as specificity and inclusion of strengths and improvement points. Differences in specificity between roles were tested with Mann–Whitney U tests with a Bonferroni correction (α?=?0.003).

Results: Supervisors (n?=?126) provided residents (n?=?120) with feedback (591 times). Feedback was provided on the intended roles, most frequently on Scholar (78%) and Communicator (71%); least on Manager (47%), and Collaborator (56%). Strengths (78%) were mentioned more frequently than improvement points (52%), which were lacking in 40% of the feedback on Manager, Professional, and Collaborator. Feedback on Scholar was more frequently (p?=?0.000) and on Reflective Professional was less frequently (p?=?0.003) specific.

Discussion and conclusion: Assigning roles to authentic situations guides supervisors in providing feedback on different CanMEDS roles. We recommend additional supervisor training on how to observe and evaluate the roles.     

Simulation-based mastery learning in medical students: Skill retention at 1-year follow up

Context: Deficits in basic skill performance and long-term skill retention among medical students and novice doctors are a persistent problem. This controlled study tested whether the addition of a mastery learning component to simulation-based teaching is associated with long-term retention and performance of peripheral venous catheter insertion.

Methods: Fourth-year medical students were assigned to receive either the control (simulation without mastery learning, n?=?131) or the intervention (simulation?+?mastery learning, n?=?133) instruction in peripheral venous catheter insertion. Performance was assessed at one year post-instruction. Eighty-four students from the control group and 71 from the intervention group participated in the assessment.

Results: Students who received the mastery learning instruction achieved higher overall test scores than did controls (median mastery learning score: 20.0, IQR 2.0; median control score 19.0, IQR 3.0; Mann–Whitney U test, p?<?0.001, effect size d?=?0.82). Pass rates also differed significantly between the groups, with 74.5% (n?=?53) of the intervention group passing compared with 33% (n?=?28) of the control group (p?<?0.001).

Conclusions: Mastery learning is an effective means of teaching practical skills to medical students, and is associated with higher scores at a 1-year follow up.     

Quantifying error in OSCE standard setting for varying cohort sizes: A resampling approach to measuring assessment quality

Background: The use of the borderline regression method (BRM) is a widely accepted standard setting method for OSCEs. However, it is unclear whether this method is appropriate for use with small cohorts (e.g. specialist post-graduate examinations).

Aims and methods: This work uses an innovative application of resampling methods applied to four pre-existing OSCE data sets (number of stations between 17 and 21) from two institutions to investigate how the robustness of the BRM changes as the cohort size varies. Using a variety of metrics, the ‘quality’ of an OSCE is evaluated for cohorts of approximately n?=?300 down to n?=?15. Estimates of the standard error in station-level and overall pass marks, R² coefficient, and Cronbach’s alpha are all calculated as cohort size varies.

Results and conclusion: For larger cohorts (n?>?200), the standard error in the overall pass mark is small (less than 0.5%), and for individual stations is of the order of 1–2%. These errors grow as the sample size reduces, with cohorts of less than 50 candidates showing unacceptably large standard error. Alpha and R² also become unstable for small cohorts. The resampling methodology is shown to be robust and has the potential to be more widely applied in standard setting and medical assessment quality assurance and research.     

The impact of informed self-assessment on the development of medical students’ learning goals

Purpose: This study investigates the contributions of self-assessment (SA) and external feedback on the development of learning goals (LG) and the influence on LG recall and implementation in medical students.

Methods: Following a standardized patient (SP) assessment, 168 pre-clinical medical students completed a SA, received SP feedback and created a LG. LG were categorized by source. Two weeks later, students recalled LG and described implementation. Chi-squared analyses were used to test the associations.

Results: SA influenced LG for 82.8% of students whereas SP feedback influenced LG for 45.9%. Students rarely generated LG based on SA when they received discordant feedback (5.4%), but sometimes incorporated feedback discordant from their SA into LG (14.9%). Students who created LG based on SP feedback were more likely to recall LG than those who created LG based on SA, 89.7 versus 67.6%, p?2(1)?=?5.3, p?=?0.017. Students who reported receiving effective feedback were more likely to implement their LG than those reporting adequate feedback, 60.9 versus 37.9%, χ²(1)?=?8.0, p?=?0.01.

Conclusions: SA is an essential part of goal setting and subsequent action. Perception of feedback plays a crucial role in LG implementation.     

Evaluation of medical record quality and communication skills among pediatric interns after standardized parent training history-taking in China

Objective: The effect of using standardized parent training history-taking on the quality of medical records and communication skills among pediatric interns was determined.

Methods: Fifth-year interns who were undertaking a pediatric clinical practice rotation were randomized to intervention and control groups. All of the pediatric interns received history-taking training by lecture and bedside teaching. The pediatric interns in the intervention group also received standardized parent history-taking training. The following two outcome measures were used: the scores of medical records, which were written by the pediatric interns after history-taking from real parents of pediatric patients; and the communication assessment tool (CAT) assessed by real parents.

Results: The general information, history of present illness (HPI), past medical history, personal history, family history, diagnosis, diagnostic analysis, and differential diagnosis scores in the intervention group were significantly higher than the control group (p?Conclusions: Standardized parent training history-taking is effective in improving the quality of medical records by pediatric interns. Standardized parent training history-taking is a superior teaching tool for clinical reasoning ability, as well as communication skills in clinical pediatric practice.     

Group observed structured encounter (GOSCE) for third-year medical students improves self-assessment of clinical communication

Objectives: This study assesses the effectiveness of a GOSCE in teaching medical students clinical communication, as well as group collaboration and peer feedback.

Methods: The GOSCE was administered during the Internal Medicine clerkship. Groups consisted of 4–6 students and one faculty member. Students completed pre- and post-GOSCE surveys to assess confidence in clinical communication and a GOSCE evaluation to rate the overall experience. Pre- and post-GOSCE program survey scores were compared, and the mean score and standard deviation of the GOSCE evaluation was calculated.

Results: Students perceived improvement in their general (Mean 4.49–4.57, p?p?p?Conclusions: A GOSCE is a valuable resource for use in formative assessment of clinical communication, and it offers the benefit of group collaboration and peer feedback. These findings support the broader use of GOSCEs in undergraduate medical education.     

The impact of medical school assessment on preparedness for practice

Purpose: As assessment is known to drive learning, this paper looks at the relationship between assessment practice across UK medical schools and graduates preparedness for practice.

Materials and methods: It uses data on written and practical assessment at each medical school and the association with students’ self-reported preparedness for working as a foundation doctor on graduation, and in particular the preparation related to clinical skills.

Results and conclusions: A negative correlation (β=??0.003, p?<?0.001) was observed between total duration of written assessment and preparedness, while a positive relationship (β?=?0.461, p?<?0.001) was seen between “adequately prepared” and the proportion of all assessment time focusing on practical skills. This suggests that graduates from medical schools with a greater emphasis on practical skills in their assessment plan are better prepared to practice as a junior doctor on gradation; something that may be of relevance when designing a national licensing examination.     

Peer-assisted learning in a student-run free clinic project increases clinical competence

Background: Germany's first student-run free clinic (SRFC) for medically underserved patients has been established at the medical faculty of the Goethe-University, Frankfurt/Germany. Participating students are educated in a Peer Assisted Learning program (PAL). Little is known about the effectiveness of PAL in SRFCs.

Methods: We conducted a randomized, controlled, prospective study involving 50 participants. Students were either tested before or after receiving PAL. Knowledge and skill level were measured by theoretical and practical tests. In addition, curricular Objective Structured Clinical Examination (OSCE) results were compared between the groups.

Results: Students receiving PAL had significantly better results in theoretical (p?<?0.001) and practical (p?<?0.001) tests, as well as in the OSCE (p?<?0.01). A control test showed no significant difference (p?=?0.205) indicating similar prerequisites between the groups.

Conclusion: Improved results of the study group indicate an increase in the clinical knowledge and skills. PAL appears to be suitable for the training of basic medical skills and family medicine related knowledge and similar teaching projects could be based on it at other SRFCs.     

Health providers’ communication skills training affects hypertension outcomes

Objective: Hypertension is a common cardiovascular risk factor within the Iranian population, and this may be improved through changes in lifestyle. We aimed at improving hypertension outcomes and health literacy skills among hypertensive patients through communication skills training targeting health providers.

Methods: A randomized controlled trial method was used to enroll 35 health providers and 240 hypertensive patients attending community-based healthcare practices in the Mashhad, Iran. We evaluated the effects of a communication skills intervention for primary care providers compared to usual care controls, on the hypertension outcomes, patient medication adherence, and self-efficacy, assessed at baseline and post-intervention. Bivariate analysis and the regression model were used to assess whether the health provider training improved outcomes.

Results: Majority of participating patients were female (77.3%), less than high school education (80.3%), married (82.3%), and low income (82.3%), with mean age of 37?years. Following the educational intervention, there was a significant improvement (p?Conclusions: The brief communication skills training for health care providers appear to be an efficient way to improve patient–provider communication skills and hypertension outcome among patients with uncontrolled BP.     

Training junior faculty to become clinical teachers: The value of personalized coaching

Abstract

Background: Junior clinical faculty require institutional support in the acquisition of feedback and clinical supervision skills of trainees. We tested the effectiveness of a personalized coaching versus guided self-reflection format of a faculty development program at improving faculty skills and self-efficacy.

Methods: Participants were evaluated both before and after the program using a four-station Objective Structured Teaching Exercise (OSTE). A gain-score analysis, one-way ANOVA, and paired t-tests were used to evaluate both groups. The impact on the learning environment was measured by resident ratings of the Maastricht Clinical Teaching Questionnaire.

Results: One hundred and twenty-seven participants completed the study over a three-year period. Both groups had significant improvements in self-efficacy. Participants in the coaching group demonstrated superior performance in encouraging learner self-reflection, teaching effectiveness, verifying learner understanding, exploring feelings/needs, and defining learning objectives. Over a 5-year period, the overall institutional learning climate significantly improved concerning faculty role-modeling, coaching, articulation, and explorations skills.

Conclusion: Offering a contextualized faculty-development program using OSTEs that provides multiple opportunities for feedback and is focused on creating a community of practice is an effective method to facilitate the transfer of skills to the clinical environment, supports teacher identity development, and favorably impacts the learning climate.     

Comparison of faculty assessment and students’ self-assessment of performance during clinical case discussions in a pharmacotherapy capstone course

Objectives: The primary objective of this study was to compare faculty assessment and third year students' self-assessment of performance in clinical case discussions. The secondary objective was to evaluate if student characteristics influence self-assessments.

Methods: This retrospective analysis compared faculty and student self-assessment scores for two clinical case discussions using Spearman’s correlation and Wilcoxon’s signed ranks test. Chi-squared test was used to compare frequency of faculty and student self-assessments indicating the highest possible rating for the pooled score and for each individual component. The pooled score included three individual components: level of engagement, quality of contribution, and professionalism.

Results: Pooled faculty and student self-assessments correlated for both the first (r?=?0.41, p?r?=?0.35; p?p?=?0.25) and second (58.6% vs. 47.4%, p?=?0.05) clinical case discussions. Student characteristics (age, gender, and grade point average at graduation) did not influence self-assessments.

Conclusions: Students’ self-assessment correlated with faculty assessment of performance during clinical case discussions. Increased use of self-assessments for professional development in pharmacy and other healthcare professional curricula should be considered.