Teaching and Assessing Resident Competence in Practice-based Learning and Improvement

We designed, implemented, and evaluated a 4-week practice-based learning and improvement (PBLI) elective. Eleven internal medicine residents from 2 separate residency programs participated in the PBLI elective and 22 other residents comprised a comparison group. Residents in each group had similar pretest Quality Improvement Knowledge Application Tool scores; but after the PBLI elective, participant scores were significantly higher. Also, participants' self-assessed ratings of PBLI skills increased after the rotation and remained elevated 6 months afterward. In this curriculum, residents completed a project to improve patient care and demonstrated their knowledge on an evaluation tool in a way that was superior to nonparticipants.     

A Self-instructional Model to Teach Systems-based Practice and Practice-based Learning and Improvement

Background  When mandated as resident competencies in 1999, systems-based practice (SBP) and practice-based learning and improvement (PBLI) were new concepts to many. Objective  To describe and evaluate a 4-week clinical elective (Achieving Competence Today—ACT) to teach residents SBP and PBLI. Design  ACT consisted of a four-week active learning course and follow-up teaching experience, guided and supported by web-based materials. The curriculum included readings, scheduled activities, work products including an improvement project, and weekly meetings with a non-expert preceptor. The evaluation used a before–after cross-comparison of ACT residents and their peers. Participants  Seventy-eight residents and 42 faculty in 18 US Internal Medicine residency programs participated between 2003 and 2005. Results and Main Measurements  All residents and faculty preceptors responded to a knowledge test, survey of attitudes, and self-assessment of competency to do 15 tasks related to SBP/PBLI. All measures were normalized to a 100-point scale. Each program’s principal investigator (PI) identified aspects of ACT that were most and least effective in enhancing resident learning. ACT residents’ gains in knowledge (4.4 on a 100-point scale) and self-assessed competency (11.3) were greater than controls’ (−1.9, −8.0), but changes in attitudes were not significantly different. Faculty preceptors’ knowledge scores did not change, but their attitudes became more positive (15.8). PIs found a ready-to-use curriculum effective (rated 8.5 on a 10-point scale). Conclusions  ACT increased residents’ knowledge and self-assessment of their own competency and raised faculty’s assessment of the importance of residents’ learning SBP/PBLI. Faculty content expertise is not required for residents to learn SBP/PBLI.     

Resident Self-Assessment and Self-Reflection: University of Wisconsin-Madison’s Five-Year Study

BACKGROUND  Chart review represents a critical cornerstone for practice-based learning and improvement in our internal medicine residency program. OBJECTIVE  To document residents’ performance monitoring and improvement skills in their continuity clinics, their satisfaction with practice-based learning and improvement, and their ability to self-reflect on their performance. DESIGN  Retrospective longitudinal design with repeated measures. PARTICIPANTS  Eighty Internal Medicine residents abstracted data for 3 consecutive years from the medical records of their 4,390 patients in the University of Wisconsin-Madison (UW) Hospital and Clinics and William S. Middleton Veterans Administration (VA) outpatient clinics. MEASUREMENT  Logistic modeling was used to determine the effect of postgraduate year, resident sex, graduation cohort, and clinic setting on residents’ “compliance rate” on 17 nationally recognized health screening and chronic disease management parameters from 2003 to 2007. RESULTS  Residents’ adherence to national preventive and chronic disease standards increased significantly from intern to subsequent years for administering immunizations, screening for diabetes, cholesterol, cancer, and behavioral risks, and for management of diabetes. Of the residents, 92% found the chart review exercise beneficial, with 63% reporting gains in understanding about their medical practices, 26% reflecting on specific gaps in their practices, and 8% taking critical action to improve their patient outcomes. CONCLUSIONS  This paper provides support for the feasibility and practicality of this limited-cost method of chart review. It also directs our residency program’s attention in the continuity clinic to a key area important to internal medicine training programs by highlighting the potential benefit of enhancing residents’ self-reflection skills.     

Resident utilization of information technology

OBJECTIVE: To determine if a simple educational intervention can increase resident physician literature search activity. DESIGN: Randomized controlled trial. SETTING: University hospital-based internal medicine training program. PATIENTS/PARTICIPANTS: Forty-eight medical residents rotating on the general internal medicine service. INTERVENTIONS: One-hour didactic session, the use of well-built clinical question cards, and practical sessions in clinical question building. MEASUREMENTS AND MAIN RESULTS: Objective data from the library information system that included the number of log-ons to medline, searching volume, abstracts viewed, full-text articles viewed, and time spent searching. Median search activity as measured per person per week (control vs intervention): number of log-ons to medline (2.1 vs 4.4, P <.001); total number of search sets (24.0 vs 74.2, P <.001); abstracts viewed (5.8 vs 17.7, P=.001); articles viewed (1.0 vs 2.6, P=.005); and hours spent searching (0.8 vs 2.4, P <.001). CONCLUSIONS: A simple educational intervention can markedly increase resident searching activity.     

Trends in Primary Care Clinician Perceptions of a New Electronic Health Record

Background  Clinician perceptions of a newly implemented electronic health record play an important role in its success or failure. Objective  To measure changes in primary care clinician attitudes toward an electronic health record during the first year following implementation. Design  Longitudinal survey. Participants  86 primary care clinicians surveyed between December 2006 and January 2008. Measurements  Perceived impact on overall quality of care, patient safety, communication, and efficiency at 1, 3, 6, and 12 months following implementation. Results  Response rates for months 1, 3, 6, and 12 were 92%, 95%, 90%, and 82%, respectively. The proportion of clinicians agreeing that the EHR improved the overall quality of care (63% to 86%; p < 0.001), reduced medication-related errors (72% to 81%; p = 0.03), improved follow-up of test results (62% to 87%; p < 0.001), and improved communication among clinicians (72% to 93%; p < 0.001) increased from month 1 to month 12. During the same time period, a decreasing proportion of clinicians agreed that the EHR reduced the quality of patient interactions (49% to 33%; p = 0.001), resulted in longer patient visits (68% to 51%; p = 0.001), and increased time spent on medical documentation (78% to 68%; p = 0.006). Significant improvements in perceptions related to test result follow-up were first detected at 6 months, while those related to overall quality, efficiency, and communication were first identified at 12 months. Conclusions  Primary care clinicians report increasingly positive perceptions of a new electronic health record within 1 year of implementation across a spectrum of domains of care. This study was funded by grants from the Agency for Healthcare Research and Quality (1 R01 HS 015226-01) and the National Library of Medicine (2 T15 LM 07092-16).     

Primary Care Resident Perceived Preparedness to Deliver Cross-cultural Care: An Examination of Training and Specialty Differences

Objective Previous research has shown that resident physicians report differences in training across primary care specialties, although limited data exist on education in delivering cross-cultural care. The goals of this study were to identify factors that relate to primary care residents’ perceived preparedness to provide cross-cultural care and to explore the extent to which these perceptions vary across primary care specialties. Design Cross-sectional, national mail survey of resident physicians in their last year of training. Participants Eleven hundred fifty primary care residents specializing in family medicine (27%), internal medicine (23%), pediatrics (26%), and obstetrics/gynecology (OB/GYN) (24%). Results Male residents as well as those who reported having graduated from U.S. medical schools, access to role models, and a greater cross-cultural case mix during residency felt more prepared to deliver cross-cultural care. Adjusting for these demographic and clinical factors, family practice residents were significantly more likely to feel prepared to deliver cross-cultural care compared to internal medicine, pediatric, and OB/GYN residents. Yet, when the quantity of instruction residents reported receiving to deliver cross-cultural care was added as a predictor, specialty differences became nonsignificant, suggesting that training opportunities better account for the variability in perceived preparedness than specialty. Conclusions Across primary care specialties, residents reported different perceptions of preparedness to deliver cross-cultural care. However, this variation was more strongly related to training factors, such as the amount of instruction physicians received to deliver such care, rather than specialty affiliation. These findings underscore the importance of formal education to enhance residents’ preparedness to provide cross-cultural care.     

Combining Quality Improvement and Geriatrics Training: The Nursing Home Polypharmacy Outcomes Project

To examine sustained effects of an educational intervention, the authors repeated a successful quality improvement (QI) project on medication safety and cost effectiveness. In October 2007 and August 2008, the facility leadership and geriatrics faculty identified all patients receiving nine or more medications (polypharmacy cohort) in a 170-bed teaching nursing home. They then taught Geriatric Medicine fellows (n = 12 in 2007, 11 in 2008) to (a) systematically collect medication data; (b) generate medication recommendations (stop, taper, or continue) based on expert criteria (Beers criteria) or drug–drug interaction programs; (c) discuss recommendations with patients’ attending physicians; and (d) implement approved recommendations. Over the two projects, the polypharmacy cohorts demonstrated decreased potentially inappropriate medications (odds ratio [OR] = .78, 95% confidence interval [95% CI] [0.69, 0.88], p < .001), contraindicated medications (OR = .63, 95% CI [0.47, 0.85], p = .002) and medication costs (OR = .97, 95% CI [0.96, 0.99], p < .001). Findings suggest that programs planning educational QI projects for trainees may benefit from a multiyear approach to maximize clinical and educational benefits.     

Effectiveness of a 1-Year Resident Training Program in Clinical Research: A Controlled Before-and-After Study

Background To increase the number of clinician scientists and to improve research skills, a number of clinical research training programs have been recently established. However, controlled studies assessing their effectiveness are lacking. Objective To investigate the effectiveness of a 1-year resident training program in clinical research. Design Controlled before-and-after study. The training program included a weekly class in clinical research methods, completion of a research project, and mentorship. Participants Intervention subjects were 15 residents participating in the 1-year training program in clinical research. Control subjects were 22 residents not participating in the training program. Measurements and Main Results Assessments were performed at the beginning and end of the program. Outcomes included methodological research knowledge (multiple-choice progress test), self-assessed research competence, progress on publications and grant applications, and evaluation of the program using quantitative and qualitative methods. Results Intervention subjects and controls were well matched with respect to research experience (5.1 ± 2.2 vs 5.6 ± 5.8 years; p = .69). Methodological knowledge improved significantly more in the intervention group compared to the control group (effect size = 2.5; p < .001). Similarly, self-assessed research competence increased significantly more in the intervention group (effect size = 1.1; p = .01). At the end of the program, significantly more intervention subjects compared to controls were currently writing journal articles (87% vs 36%; p = .003). The intervention subjects evaluated the training program as highly valuable for becoming independent researchers. Conclusions A 1-year training program in clinical research can substantially increase research knowledge and productivity. The program design makes it feasible to implement in other academic settings.     

Creating a quality improvement elective for medical house officers

The Accreditation Council on Graduate Medical Education (ACGME) requires that house officers demonstrate competencies in "practice-based learning and improvement" and in "the ability to effectively call on system resources to provide care that is of optimum value." Anticipating this requirement, faculty at a Boston teaching hospital developed a 3-week elective for medical house officers in quality improvement (QI). The objectives of the elective were to enhance residents' understanding of QI concepts, their familiarity with the hospital's QI infrastructure, and to gain practical experience with root-cause analysis and QI initiatives. Learners participated in three didactic seminars, joined hospital-based QI activities, conducted a root-cause analysis, and completed a QI project under the guidance of a faculty mentor. The elective enrolled 26 residents in 3 years. Sixty-three percent of resident respondents said that the elective increased their understanding of QI in health care; 88% better understood QI in their own institution.     

Leveraging Computerized Sign-Out to Increase Error Reporting and Addressing Patient Safety in Graduate Medical Education

SUMMARY Setting Electronic reporting systems are important components of the patient-safety movement but have been problematic particularly in capturing information from physicians. Objective In an attempt to increase error reporting in our community hospital residency program, we devised a convenient safety log integrated into the computerized resident-patient sign-out. Measurements and Main Results Safety-log events are discussed in weekly safety reports with emphasis on developing a safety culture and preventing further events. We report our experience implementing the safety log. The program quadrupled our contribution to the hospital quality assurance system and has led to significant system innovations. Challenges have included sharing information and improving accountability without humiliating participants or alienating co-workers, as well as sustaining the project over time. Conclusion Residents are uniquely placed to provide insight into hospital systems. An error-reporting system based in a resident sign-out can leverage this role into an important tool for safety education and care improvement.     

The Science of Learning and Art of Education in Cardiology Fellowship

The science of learning, bolstered by the foundational principles of adult learning, has evolved to allow for a more sophisticated understanding of how humans acquire knowledge. To optimize learning outcomes, cardiology educators should be familiar with these concepts and apply them routinely when teaching trainees. This paper presents an overview of the neurobiology of learning and adult learning principles and offers examples of ways in which this science can be applied in cardiology fellowships. Both fellows and educators benefit from the science of learning and its artistic application to education.     

Attitudes,practices and perceived barriers to hepatitis C screening among medical residents at a large urban academic medical center

Infections with hepatitis C virus (HCV) are increasing among adolescents and adults born after 1965. Screening strategies may need to be adapted for this changing population. We surveyed trainees in different specialties about attitudes and practices related to HCV screening and identified specific barriers to screening across various healthcare settings. Constraints related to health system resources and the provider's role were among the most common barriers cited across specialties, but paediatrics residents also cited barriers specific to their population, which can likely be addressed with targeted education.     

Use of Multidisciplinary Rounds to Simultaneously Improve Quality Outcomes,Enhance Resident Education,and Shorten Length of Stay

BACKGROUND Hospital-based clinicians and educators face a difficult challenge trying to simultaneously improve measurable quality, educate residents in line with ACGME core competencies, while also attending to fiscal concerns such as hospital length of stay (LOS). OBJECTIVE The purpose of this study was to determine the effect of multidisciplinary rounds (MDR) on quality core measure performance, resident education, and hospital length of stay. DESIGN Pre and post observational study assessing the impact of MDR during its first year of implementation. SETTING The Norwalk Hospital is a 328-bed, university-affiliated community teaching hospital in an urban setting with a total of 44 Internal Medicine residents. METHODS Joint Commission on Accreditation of Healthcare Organizations (JCAHO) core measure performance was obtained on a monthly basis for selected heart failure (CHF), pneumonia, and acute myocardial infarction (AMI) measures addressed on the general medical service. Resident knowledge and attitudes about MDR were determined by an anonymous questionnaire. LOS and monthly core measure performance rates were adjusted for patient characteristics and secular trends using linear spline logistic regression modeling. RESULTS Institution of MDR was associated with a significant improvement in quality core measure performance in targeted areas of CHF from 65% to 76% (p < .001), AMI from 89% to 96% (p = .004), pneumonia from 27% to 70% (p < .001), and all combined from 59% to 78% (p < .001). Adjusted overall monthly performance rates also improved during MDR (odds ratio [OR] 1.09, CI 1.06–1.12, p < .001). Residents reported substantial improvements in core measure knowledge, systems-based care, and communication after institution of MDR (p < .001). Residents also agreed that MDR improved efficiency, delivery of evidence-based care, and relationships with involved disciplines. Adjusted average LOS decreased 0.5 (95% CI 0.1–0.8) days for patients with a target core measure diagnosis of either CHF, pneumonia, or AMI (p < .01 ) and by 0.6 (95% CI 0.5–0.7) days for all medicine DRGs (p < .001). CONCLUSIONS Resident-centered MDR is an effective process using no additional resources that simultaneously improves quality of care while enhancing resident education and is associated with shortened length of stay. This study was presented in part in workshop and oral format at the 27th Society of General Internal Medicine Annual Meeting, May 12–15, 2004, Chicago, IL     

Development and Implementation of a Quality Improvement Process for Echocardiographic Laboratory Accreditation

We describe our process for quality improvement (QI) for a 3‐year accreditation cycle in echocardiography by the Intersocietal Accreditation Commission (IAC) for a large group practice. Echocardiographic laboratory accreditation by the IAC was introduced in 1996, which is not required but could impact reimbursement. To ensure high‐quality patient care and community recognition as a facility committed to providing high‐quality echocardiographic services, we applied for IAC accreditation in 2010. Currently, there is little published data regarding the IAC process to meet echocardiography standards. We describe our approach for developing a multicampus QI process for echocardiographic laboratory accreditation during the 3‐year cycle of accreditation by the IAC. We developed a quarterly review assessing (1) the variability of the interpretations, (2) the quality of the examinations, (3) a correlation of echocardiographic studies with other imaging modalities, (4) the timely completion of reports, (5) procedure volume, (6) maintenance of Continuing Medical Education credits by faculty, and (7) meeting Appropriate Use Criteria. We developed and implemented a multicampus process for QI during the 3‐year accreditation cycle by the IAC for Echocardiography. We documented both the process and the achievement of those metrics by the Echocardiography Laboratories at the Ochsner Medical Institutions. We found the QI process using IAC standards to be a continuous educational experience for our Echocardiography Laboratory physicians and staff. We offer our process as an example and guide for other echocardiography laboratories who wish to apply for such accreditation or reaccreditation.