Integrating an evidence-based medicine rotation into an internal medicine residency program.

PURPOSE: To measure the impact of a resident focused evidence-based medicine (EBM) educational intervention on EBM knowledge of residents and students, to assess its feasibility, and to evaluate residents' attitudes regarding this rotation. METHOD: In 2002, based on the EBM user and EBM practitioner model, the authors designed the EBM elective rotation and conducted a controlled trial of its implementation in the internal medicine residency program in three teaching hospitals affiliated with the University at Buffalo, New York. The intervention group (one hospital, 17 medical students and residents) received a multifaceted intervention. In the control group (two hospitals, 23 medical students and residents), there was no curriculum change. The effectiveness in a pre- and post-test was assessed using the English version of the Berlin Questionnaire. A survey of all internal medicine residents (n = 119) was conducted to evaluate their attitudes toward the EBM elective rotation. RESULTS: In the intervention group, knowledge improved slightly, but not significantly (.71 on a scale ranging from 0-15 on the Berlin questionnaire, p =.3). The mean score in the control group decreased significantly (1.65, p =.005). The difference in change scores between the two groups was significant even after adjustment for covariates (2.52, p =.006). Residents (response rate 83%) had positive attitudes regarding the rotation. CONCLUSION: An EBM elective rotation was successfully integrated into a residency program. This multifaceted educational approach with an "on-the-ward" EBM resident, may improve the EBM knowledge and skills of targeted students and residents.     

Measuring outcomes of undergraduate medical education: residency directors' ratings of first-year residents.

PURPOSE: There is a growing recognition of the need to show the relationship between undergraduate medical education (UME) and achievements during residency. This study provides reliability and validity evidence for a residency rating scale as well as a method for gathering comparison information about first-year residents. METHOD: A 25-item rating scale measuring important areas of physician functioning was mailed to residency directors of 485 graduates of the 1998-2000 classes of the University of Kansas School of Medicine. The same rating scale was sent to residency directors for a comparison sample of 251 graduates of other U.S. medical schools who were residents at the University of Kansas Medical Center. Each item on the rating scale was rated on a five-point Likert scale. Principal-components analysis, correlational analyses, internal consistency reliability analysis, and mean comparisons were used to provide evidence of reliability and validity. RESULTS: A total of 382 (82%) usable rating scales were returned. A principal-components analysis extracted five factors that accounted for 86% of the variance. The final factors were (1) interpersonal communication, (2) clinical skills, (3) population-based health care, (4) record-keeping skills, and (5) critical appraisal skills. The internal consistency of the entire scale was.98, with coefficients for the five factors ranging from.92 to.97. The correlations between the five factors and measures of undergraduate performance ranged from.21 to.49. Group analyses revealed that residents with high GPAs and USMLE Step 1 and Step 2 scores tended to be rated higher than those with lower scores. CONCLUSIONS: The rating scale demonstrated adequate reliability and validity and showed that residency directors' ratings are a useful outcome measure for UME performance.     

Aging Q3: an initiative to improve internal medicine residents' geriatrics knowledge, skills, and clinical performance

A growing number of older adults coupled with a limited number of physicians trained in geriatrics presents a major challenge to ensuring quality medical care for this population. Innovations to incorporate geriatrics education into internal medicine residency programs are needed. To meet this need, in 2009, faculty at the Medical University of South Carolina developed Aging Q(3)-Quality Education, Quality Care, and Quality of Life. This multicomponent initiative recognizes the need for improved geriatrics educational tools and faculty development as well as systems changes to improve the knowledge and clinical performance of residents. To achieve these goals, faculty employ multiple intervention strategies, including lectures, rounds, academic detailing, visual cues, and electronic medical record prompts and decision support. The authors present examples from specific projects, based on care areas including vision screening, fall prevention, and caring for patients with dementia, all of which are based on the Assessing Care of Vulnerable Elders quality indicators. The authors describe the principles driving the design, implementation, and evaluation of the Aging Q(3) program. They present data from multiple sources that illustrate the effectiveness of the interventions to meet the knowledge, skill level, and behavior goals. The authors also address major challenges, including the maintenance of the teaching and modeling interventions over time within the context of demanding primary care and inpatient settings. This organized, evidence-based approach to quality improvement in resident education, as well as faculty leadership development, holds promise for successfully incorporating geriatrics education into internal medicine residencies.     

Insights from outstanding rural internal medicine residency rotations at the University of Washington.

PURPOSE: Despite being well suited to provide the breadth of care needed in rural areas, few general internists become rural physicians. Little formal rural residency training is available and no formal curricula exist. For over 25 years the University of Washington School of Medicine has provided elective WWAMI (Washington, Wyoming, Alaska, Montana, Idaho) rural residency rotations to expose residents to the rewards and challenges of rural practice. This study identified the characteristics of outstanding rural residency rotations. METHOD: The key preceptors at three outstanding rural residency sites were interviewed about their experiences, teaching strategies, and opinions about curriculum. Their responses were categorized. Seven university-based residents and eight training at WWAMI sites recorded and rated the value of over 1,500 learning encounters. RESULTS: The preceptors agreed that outstanding rotations were led by enthusiastic preceptors who served as role models for excellence. These preceptors provided residents with meaningful responsibilities and emphasized independent decision making based on the history and physical examination. They stressed supervised independence and self-directed learning with frequent structured feedback for residents. The residents rated the learning value of patient encounters in rural locations significantly higher than that of those in university clinics. CONCLUSIONS: Exceptional rural residency experiences involve excellent role models who provide meaningful responsibility and emphasize core skills using a learner-centered approach. Rural training experiences should be supported, and the suggestions of outstanding preceptors should be used to develop and disseminate a curriculum that will better prepare residents for rural practice.     

Evaluation of procedural skills of internal medicine residents

In 1983 the American Board of Internal Medicine mandated that training programs assess residents in procedural skills prior to board certification. Beginning in 1984, the University of Hawaii developed an assessment and testing program that consists of a three-phase evaluation process for qualifying internal medicine residents in basic procedural skills: an observation period spanning the three-year residency, a 100-question multiple-choice examination, and a slide-identification examination involving simple identification of body fluid elements. From 1984 through 1987 the mean scores of both examinations were analyzed for each level of residency and for each year of testing to assess whether curriculum changes were effective. There was a statistically significant improvement (p less than .005) in performance on both the multiple-choice and the slide-identification examinations as the residents progressed through the program. No difference in performance was found for the multiple-choice examination between calendar years, but a significant difference (p less than .01) was found for the slide-identification examination between calendar years. Despite perceptions by the faculty that their increased emphasis on the learning of procedural skills has been successful, the multiple-choice examination results over time do not support this belief. These findings suggest that residents need more structured curriculum guidelines to aid their learning and faculty members require better direction in the educational goals for procedural skills training.     

Part-time residency training in internal medicine: analysis of a ten-year experience.

PURPOSE: To evaluate a ten-year experience (1983-1993) with a part-time residency curriculum. METHOD: In 1994, the authors analyzed the curriculum through interviews with graduates of a part-time residency track, surveys of faculty and graduates of a full-time residency program, and a quantitative comparison of faculty evaluations of those part-time and full-time residents. RESULTS: Both participants and full-time residents supported the part-time track and reported no adverse effect on the residency program as a whole. Analysis of faculty evaluations found that part-time residents scored significantly higher with respect to clinical skills (p = .0005) and humanistic skills (p = .0001), while there was no difference between the groups in leadership or teaching skills. CONCLUSIONS: This part-time residency curriculum provided a highly useful program track for a group of internal medicine residents with concomitant obligations, allowing them to complete their training in an uninterrupted fashion. The part-time structure did not adversely affect clinical competence and may have fostered humanistic attributes. The authors believe that this form of curriculum deserves wider consideration in residency training.     

A pilot randomized, controlled trial of a longitudinal residents-as-teachers curriculum.

PURPOSE: To determine whether a longitudinal residents-as-teachers curriculum improves generalist residents' teaching skills. METHOD: From May 2001 to February 2002, 23 second-year generalist residents in four residencies affiliated with the University of California, Irvine, College of Medicine, completed a randomized, controlled trial of a longitudinal residents-as-teachers program. Thirteen intervention residents underwent a 13-hour curriculum during one-hour noon conferences twice monthly for six months, practicing teaching skills and receiving checklist-guided feedback. In a 3.5-hour, eight-station objective structured teaching examination (OSTE) enacted and rated by 15 senior medical students before and after the curriculum, two trained, blinded raters independently assessed each station with detailed, case-specific rating scales (rating scale reliability = 0.96, inter-rater reliability = 0.78). RESULTS: The intervention and control groups were similar in academic performance, specialty distribution, and gender (chi(2) = 0.434, p =.81). On a five-point Likert scale (5 = best teaching skills), intervention and control residents showed similar mean pretest OSTE scores (2.83 vs. 2.88, p =.736). The intervention group improved their mean overall OSTE scores 22.3% (more than two standard deviations) from 2.83 (pretest) to 3.46 (post-test; p <.0005; 95% CI 0.53 to 0.72). Intervention residents also improved significantly on six of eight OSTE stations. Within the control group, no pretest-to-post-test change achieved statistical significance. Mann-Whitney and Wilcoxon signed-rank tests confirmed these results. CONCLUSIONS: Generalist residents randomly assigned to receive a 13-hour longitudinal residents-as-teachers curriculum consistently showed improved OSTE scores. Future research should clarify which aspects of residents-as-teachers curricula most effectively improve educational outcomes.     

Preparing graduates for the first year of residency: are medical schools meeting the need?

PURPOSE: Changes in graduate medical education associated with full implementation of the Balanced Budget Act of 1997 have required medical schools to review and revise their curricula. As limited funding increases pressures to streamline training, residencies will potentially expect an entry level of skill and competence that is greater than that which schools are currently providing. To determine whether medical school curricular requirements correlate with residency needs, this multidisciplinary pilot study investigated expectations and prerequisites for postgraduate specialty training. METHOD: A questionnaire about 100 skills and competencies expected of new first-year residents was sent to 50 U.S. residency directors from surgery, internal medicine, family medicine, pediatrics, and obstetrics-gynecology programs. Each director was asked to state expectations of a first-year resident's competence in each skill at entry to residency and after three months of training. Skills deemed most appropriately acquired in residency were also identified. Competencies included diagnosis, management, triage, interpretation of data, informatics and technology, record keeping, interpersonal communications, and manual skills. RESULTS: A total of 39 residency directors responded, including seven surgery, nine medicine, seven family medicine, eight pediatrics, and eight obstetrics-gynecology. In addition to physical examination skills, 13 competencies achieved more than 70% agreement as being entry-level skills. There was wide variability as to the relative importance of the remaining skills, with residency directors expecting to devote significant resources and time in early training to ensure competence. CONCLUSIONS: Medical schools should consider the expectations of their students' future residency directors when developing new curricula. Assuring students' competencies through focused curricular change should save both time and resources during residency.     

Hospital to home: Improving internal medicine residents' understanding of the needs of older persons after a hospital stay.

Physicians-in-training discharge many older patients from the hospital, but few have any knowledge of what happens to the patients they send home, of how discharge plans are applied, or of the difficulties patients and their families face. The authors describe a pilot program, Hospital to Home, at the University of Rochester School of Medicine and Dentistry's internal medicine residency program, which uses home visits as an educational tool in geriatrics training. The program was begun in July 2001, and 23 residents have participated. Home visits expose residents in their first-year geriatrics rotation to the elements and outcomes of discharge planning and create a heightened awareness of the needs of older persons recently discharged from the hospital. The home visits are videotaped, and the residents present a videoconference based on the visits, which are attended by internal medicine residents, family medicine residents, and medical students. The authors describe the three-part Hospital to Home program, three vignettes that highlight learning experiences, and the residents' feedback about the experience and the use of audiovisual recording for education.     

Informed consent skills in internal medicine residency: how are residents taught, and what do they learn?

PURPOSE: Obtaining informed consent is an essential skill in internal medicine (IM). The authors' informal observations and formal testing revealed deficiencies in residents' informed consent skills. This study evaluated how residents acquire informed consent skills and how informed consent skills are addressed in Canadian IM residency programs. METHOD: A questionnaire was delivered to all 16 IM program directors in Canada, asking how informed consent is taught and assessed. At the University of Saskatchewan IM residency program, residents were assessed through an objective structured clinical examination station, written examination, and a self-assessment questionnaire. RESULTS: No consistent approach to teaching or evaluating informed consent skills exists within Canadian IM programs. Program directors and residents identified informal mentoring by residents as an important learning modality. Although residents performed well in discussing procedural indications and techniques, discussing risks was inadequate. Residents focused on general and minor risks but avoided discussing serious risks and had difficulty discussing the frequency of complications. Residents lacked a structured approach to assessing capacity and often assessed only comprehension. Residents were unfamiliar with concepts such as material risk, implied consent, and therapeutic privilege. CONCLUSION: Explicit training in informed consent skills is urgently needed. Informal mentoring must be recognized as an important training method for informed consent and supported by appropriate teaching and evaluation strategies to ensure that resident-instructors do so effectively.     

Perspectives on death and dying: a study of resident comfort with End-of-life care

Background

Despite the benefits to early palliative care in the treatment of terminal illness, barriers to timely hospice referrals exist. Physicians who are more comfortable having end-of-life (EOL) conversations are more likely to refer to hospice. However, very little is known about what factors influence comfort with EOL care.

Methods

An anonymous survey was sent to all the residents and fellows at a single institution. Self-reported education, experience and comfort with EOL care was assessed. Using multivariate logistic regression analysis, variables that influenced comfort with EOL conversations were analyzed.

Results

Most residents (88.1%) reported little to no classroom training on EOL care during residency. EOL conversations during residency were frequent (50.6% reported?>?10) and mostly unsupervised (61.9%). In contrast, EOL conversations during medical school were infrequent (3.7% reported >10) and mostly supervised (78.6%). Most (54.3%) reported little to no classroom training on EOL care during medical school. Physicians that reported receiving education on EOL conversations during residency and those who had frequent EOL conversations during residency had significantly higher comfort levels having EOL conversations (p?=?0.017 and p?=?0.003, respectively). Likewise, residents that felt adequately prepared to have EOL conversations when graduating from medical school were more likely to feel comfortable (p?=?0.030).

Conclusions

Most residents had inadequate education in EOL conversation skills during medical school and residency. Despite the lack of training, EOL conversations during residency are common and often unsupervised. Those who reported more classroom training during residency on EOL skills had greater comfort with EOL conversations. Training programs should provide palliative care education to all physicians during residency and fellowship, especially for those specialties that are most likely to encounter patients with advanced terminal disease.

     

Assessing pediatrics residents' mathematical skills for prescribing medication: a need for improved training.

PURPOSE: To evaluate residents' skills in performing basic mathematical calculations used for prescribing medications to pediatric patients. METHOD: In 2001, a test of ten questions on basic calculations was given to first-, second-, and third-year residents at Miami Children's Hospital in Florida. Four additional questions were included to obtain the residents' levels of training, specific pediatrics intensive care unit (PICU) experience, and whether or not they routinely double-checked doses and adjusted them for each patient's weight. The test was anonymous and calculators were permitted. The overall score and the score for each resident class were calculated. RESULTS: Twenty-one residents participated. The overall average test score and the mean test score of each resident class was less than 70%. Second-year residents had the highest mean test scores, although there was no significant difference between the classes of residents (p =.745) or relationship between the residents' PICU experiences and their exam scores (p =.766). There was no significant difference between residents' levels of training and whether they double-checked their calculations (p =.633) or considered each patient's weight relative to the dose prescribed (p =.869). Seven residents committed tenfold dosing errors, and one resident committed a 1,000-fold dosing error. CONCLUSION: Pediatrics residents need to receive additional education in performing the calculations needed to prescribe medications. In addition, residents should be required to demonstrate these necessary mathematical skills before they are allowed to prescribe medications.     

A pilot survey study to define quality in residency education.

PURPOSE: To begin to define indicators of quality in internal medicine residency training. METHOD: In 1995, through a modified Delphi process, the Association of Program Directors in Internal Medicine's Research Committee developed a questionnaire containing 44 items (34 process and ten outcome indicators). The survey was mailed to all 418 internal medicine program directors and a convenience sample of medical residents. RESULTS: Responding at a rate of 78% (326), program directors rated several indicators as important. These included such faculty characteristics as stability, completeness, supervision, clinical skills, and teaching commitment; institutional support; amount of resident evaluation and feedback; encouragement of lifelong learning; and ability to meet its program goals. There was strong agreement between faculty and residents (r = 0.91). Items rated less important included graduates' selecting academic or generalist careers, residents' caring for elective cardiac catheterization patients, resident community service, training minorities and women, and faculty research. CONCLUSION: These results demonstrate the diversity of opinion of what defines quality in residency education and the emphasis placed on process rather than outcome indicators. To be valid, future endeavors must include all those with a stake in graduate medical education, including accrediting bodies, future employers, and patients.     

Assessing the relationship of learning approaches to workplace climate in clerkship and residency.

PURPOSE: To determine what approaches to learning are adopted by clinical clerks and residents and whether these approaches are associated with demographic factors, specialty, level of training, and perceptions of the workplace climate. METHOD: In 2001-02, medical clerks (n = 532) and residents (n = 2,939) at five medical schools in Ontario, Canada, were mailed the Workplace Learning Questionnaire. The correlation between the approaches to learning at work and perceived workplace climate and the influence of gender, age, location, residency program and level of training on outcomes were measured. RESULTS: A total of 1,642 clerks and residents responded (47%). The factor structure and reliability of the Workplace Learning Questionnaire were confirmed for these respondents. A surface-disorganized approach to learning was correlated with perception of heavy workload (r = .401, p < .001). The deep approach to learning was correlated with perception of choice-independence in the workplace and a supportive-receptive workplace (r = .32, p < .001; r = .23, p < .001). The climate factors, perception of choice-independence and supportive-receptive workplace, were correlated (r = .60, p < .001). There were significant differences among the mean scores for scales based on residency, year of training, and location of training. CONCLUSIONS: Perception of the workplace climate was associated with the approach to learning in the workplace of clerks and residents. Perception of heavy workload was associated with less effective approaches to learning. These associations varied with the residency program and the level of training.     

A communication assessment and skill-building exercise (CASE) for first-year residents.

OBJECTIVE: Good communication skills are essential for residents entering postgraduate education programs. However, these skills vary widely among medical school graduates. This pilot program was designed to create opportunities for (1) teaching essential interviewing and communication skills to trainees at the beginning of residency, (2) assessing resident skills and confidence with specific types of interview situations, (3) developing faculty teaching and assessment skills, (4) encouraging collegial interaction between faculty and new trainees, and (5) guiding residency curricular development. DESCRIPTION: During residency orientation, all first-year internal medicine residents (n = 26) at the University of Minnesota participated in the communication assessment and skill-building exercise (CASE). CASE consisted of four ten-minute stations in which residents demonstrated their communication skills in encounters with standardized patients (SPs) while faculty members observed for specific skills. Faculty and SPs were oriented to the educational purposes and goals of their stations, and received instructions on methods of providing feedback to residents. With each station, residents were provided one and a half minutes of direct feedback by the faculty observer and the SP. The residents were asked to deal with an angry family member, to counsel for smoking cessation, to set a patient-encounter agenda, and to deliver bad news. A resident's performance was analyzed for each station, and individual profiles were created. All residents and faculty completed evaluations of the exercise, assessing the benefits and areas for improvement. DISCUSSION: Evaluations and feedback from residents and faculty showed that most of our objectives were accomplished. Residents reported learning important skills, receiving valuable feedback, and increasing their confidence in dealing with certain types of stressful communication situations in residency. The activity was also perceived as an excellent way to meet and interact with faculty. Evaluators found the experience rewarding, an effective method for assessing and teaching clinical skills, a faculty development experience for themselves in learning about structured practical skills exercises, and a good way to meet new interns. The residency program director found individual resident performance profiles valuable for identifying learning issues and for guiding curricular development. Time constraints were the most frequently cited area for improvement. The exercise became feasible by collaborating with the medical school Office of Education-Educational Development and Research, whose mission is to collaborate with faculty across the continuum of medical education to improve the quality of instruction and evaluation. The residency program saved considerable time, effort, and expense by using portions of the medical school's existing student skills-assessment programs and by using chief residents and faculty as evaluators. We plan to use CASE next year with a wider variety of physician-patient scenarios for interns, and to expand the program to include beginning second- and third-year residents. Also, since this type of exercise creates powerful feedback and assessment opportunities for instructors and course directors, and because feedback was so favorable from evaluators, we will encourage participation in CASE as part of our faculty educational development program.     

The Patient Findings Questionnaire: one solution to an important standardized patient examination problem.

PURPOSE: To describe the Patient Findings Questionnaire (PFQ) and compare its scores and pass/fail decisions with those obtained from standardized patient (SP) examination checklists. METHOD: Checklists and PFQs were used to assess data acquisition by 790 second-year medical students. PFQs were composed of multiple-choice items designed to determine whether examines had acquired key historical patient information. RESULTS: At the item level, the two measurement methods yielded the same decisions about data acquisition on 88% of observed occasions. Most discrepancies (74%) involved SPs rating examinees as having elicited information when the examinee was unable to answer the associated PFQ item. At the test level, the two instruments yielded the same pass/fail decision on a large majority of occasions. CONCLUSIONS: The PFQ and checklist yielded similar data acquisition scores and decisions at the item and test levels. Replacement of the checklist with the PFQ should result in examinees' behaving in a way more consistent with recommended interviewing practices.     

Internal medicine resident perceptions of optimal training duration.

PURPOSE: To describe internal medicine residents' opinions regarding the optimal duration of internal medicine residency training, and to assess whether these opinions are associated with specific career interests. METHOD: A national cohort study was conducted during the 2005 Internal Medicine In-Training Examination (IM-ITE), which involved 382 of 388 (98.5%) U.S. internal medicine programs. A sample of 14,579 residents enrolled in three-year categorical or primary care training programs in the United States reported their opinions regarding optimal residency training duration on the IM-ITE 2005 Residents Questionnaire. Reported optimal training duration was assessed by postgraduate training year, sex, medical school location, program type, and reported career plan. RESULTS: Among the residents surveyed, 78.1% reported a three-year optimal length of internal medicine residency training, 15.3% preferred a two-year training duration, and 6.7% preferred a four-year duration. Residents planning careers in general medicine, hospital medicine, and subspecialty fields all preferred a three-year training duration (83.8%, 82.6%, and 75.9%, respectively). Residents planning subspecialty careers were more likely than those planning general or hospital medicine careers to prefer a two-year program (18.7% versus 7.4% and 8.3%). Residents planning generalist or hospitalist careers were more likely to favor a four-year program (8.9% and 9.1%, respectively) compared with residents planning subspecialty careers (5.4%). CONCLUSIONS: Most internal medicine residents endorse a three-year optimal duration of internal medicine residency training. This perspective should be considered in further national discussions regarding the optimal duration of internal medicine training.     

An innovative, longitudinal program to teach residents about end-of-life care.

At the University of California, Irvine Medical Center, an end-of-life curriculum was implemented in 2000 for an internal medicine residency utilizing a longitudinal approach that allowed residents to follow patients through their entire hospice experience. An elective home hospice rotation was developed for which third-year residents served as primary care physicians for patients at the end of life over a one-year period. Residents were supervised by faculty who were hospice medical directors. They also learned through case vignettes, quarterly meetings, textbook reading, and personal projects. From July 2000 to June 2002, residents demonstrated positive attitudes towards hospice care and recommended the rotation highly (mean 8.86 on a scale of 1-10). The rotation grew in popularity from six initial residents to ten residents the next year, and has since become a mandatory rotation for all senior residents. A 360-degree evaluation uniformly indicated positive resident performance from the hospice team (mean scores 7.56-8.69 on a 1-9 scale), family (mean scores 9.3-9.7 on a 1-10 scale) and faculty (mean scores 7.29-7.72 on a 1-9 scale). Residents were also pleased with the level of teaching (mean 8.86 on a scale of 1-10) and felt that the patient care load was "just right." Their knowledge improved by 8% (p =.0175). In conclusion, a longitudinal hospice rotation was implemented that fulfilled curricular goals without undue burden on the residents or residency program.