Measuring medical education and curriculum during orthopedic surgical residency

BACKGROUND: Evaluating residency programs requires objective assessment tools, but few are readily available. The purpose of this study was to measure education by correlating resident test scores with several measurements of educator performance. MATERIALS AND METHODS: The study group included residents and educators from a single residency program. We performed a retrospective analysis of scores from the Orthopaedic In-Training Examination collected during a 6-year period. Resident examination scores were indexed by dividing program averages by national averages to determine yearly score trends and then were correlated with educator attendance and teaching hours. Subspecialty scores were ranked to gauge residency strengths and weaknesses. Teaching hours devoted to subspecialties were compared with test scores to measure curricular emphases and to appraise teaching efficiency. RESULTS: Yearly average examination scores were proportional to national averages (P < 0.001). However, of 3436 possible educator-score associations, only 15 scores correlated highly (r > 0.9) with educators, and only 26 were significant (P < 0.05). Trend analysis put subspecialty scores in yearly perspective. Ranking was inaccurate until scores were indexed to the national average. In 2002, the distribution of 238 teaching hours ranged from 4 to 48 h for subspecialties, and 9 of 12 subspecialties were emphasized disproportionately to the examination. Teaching efficiency varied more than 10-fold by subspecialty. CONCLUSIONS: The creation of a score index helped to identify and address imbalances between teaching hours devoted to subspecialties and resident needs as evidenced by low In-Training examination scores. The present study improved educator accountability by correlating measurements of teaching and learning.     

Implementation and evaluation of a new surgical residency model

BACKGROUND: The Accreditation Council for Graduate Medical Education (ACGME) duty-hour requirements prompted program directors to rethink the organizational structure of their residency programs. Many surgical educators have expressed concerns that duty-hour restrictions would negatively affect quality of resident education. This article summarizes evaluation research results collected to study the impact of our reengineered residency program designed to preserve important educational activities while meeting duty-hour accreditation requirements. STUDY DESIGN: The traditional residency structure was redesigned to include a mixture of apprenticeship, small team, and night-float models. Impact evaluation data were collected using operative case logs, standardized test scores, quality assurance data, resident perception surveys, a faculty survey, and process evaluation measures. RESULTS: PGY1s and PGY2s enjoyed a substantial increase in operative cases. Operative cases increased overall and no resident has failed to meet ACGME volume or distribution requirements. American Board of Surgery In-Training Examination performance improved for PGY1s and PGY2s. Patient outcomes measures, including monthly mortality and number of and charges for admissions, showed no changes. Anonymously completed rotation evaluation forms showed stable or improved resident perceptions of case load, continuity, operating room teaching, appropriate level of faculty involvement and supervision, encouragement to attend conferences, and general assessment of the learning environment. A quality-of-life survey completed by residents before and after implementation of the new program structure showed substantial improvements. Faculty surveys showed perceived increases in work hours and job dissatisfaction. New physician assistant and nurse positions directly attributed to duty-hour restrictions amounted to about 0.2 full-time equivalent per resident. CONCLUSIONS: Duty-hour restrictions produce new challenges and might require additional resources but need not cause a deterioration of surgical residents' educational experience.     

Preliminary assessment of a scientific curriculum in a surgical residency program.

A structured, basic science curriculum was instituted for surgical residents of the University of Connecticut (Farmington) Integrated Residency Program during the 1990-1991 academic year in concordance with American Board of Surgery guidelines. The impact of the new program was measured by comparing performance on monthly basic science examinations, the in-training examination, and "mock" oral examinations for the 1990-1991 academic year with that of the preceding academic year. While monthly examination scores improved for the entire group of residents (67.7 vs 64.6), in-training and oral examination scores did not change significantly. Categorical residents generally demonstrated superior performance and greater improvement than did preliminary residents. Data analysis suggested that the new curriculum was an effective educational device and that university-designed monthly examinations were valid testing instruments, but there was an apparent incongruity between the goals of the curriculum and the American Board of Surgery In-Training Examination.     

An Internet-based evaluation system for a surgical residency program

BACKGROUND: Efficient and meaningful evaluation of performance is critical to the professional development of trainees in surgical residency programs. Current paper-based evaluation instruments have numerous limitations. We developed an Internet-based evaluation system to more rapidly and efficiently assess the experience of residents, faculty, and rotations. STUDY DESIGN: An on-line evaluation system was designed and implemented in October 1999. Custom evaluations were created for residents, faculty, and rotations. Evaluations were completed via the Internet site from remote locations with standard computers and standard Web browser software. Completed evaluations were automatically available on-line for review and data analysis. Data were analyzed by chi-square analysis with a probability value of less than 0.05 considered statistically significant. RESULTS: Compliance in completion of evaluations improved from 50% to 80% in the initial 6 months of implementation (p < 0.01) with the Web-based system. There was no significant difference between faculty and resident compliance. In evaluation of "ease of use," a total of 612 responses were received over this period with a total average score of 3.5 (5 point scale, 5 = strongly agree). Residents' opinions (average score, 3.69) were slightly more positive than those of faculty (average score, 3.31). Confidentiality was improved over paper-based systems by a detailed security network. CONCLUSIONS: This Internet-based evaluation system is a potentially powerful instrument for evaluating our surgical residency program and making changes to improve the educational experience in a timely and efficient manner.     

Practice management education during surgical residency

Background

Surgical education has undergone radical changes in the past decade. The introductions of laparoscopic surgery and endovascular techniques have required program directors to alter surgical training. The 6 competencies are now in place. One issue that still needs to be addressed is the business aspect of surgical practice. Often residents complete their training with minimal or no knowledge on coding of charges or basic aspects on how to set up a practice. We present our program, which has been in place over the past 2 years and is designed to teach the residents practice management.

Methods

The program begins with a series of 10 lectures given monthly beginning in August. Topics include an introduction to types of practices available, negotiating a contract, managed care, and marketing the practice. Both medical and surgical residents attend these conferences. In addition, the surgical residents meet monthly with the business office to discuss billing and coding issues. These are didactic sessions combined with in-house chart reviews of surgical coding. The third phase of the practice management plan has the coding team along with the program director attend the outpatient clinic to review in real time the evaluation and management coding of clinic visits.

Results

Resident evaluations were completed for each of the practice management lectures. The responses were recorded on a Likert scale. The scores ranged from 4.1 to 4.8 (average, 4.3). Highest scores were given to lectures concerning negotiating employee agreements, recruiting contracts, malpractice insurance, and risk management. The medical education department has tracked resident coding compliance over the past 2 years. Surgical coding compliance increased from 36% to 88% over a 12-month period. The program director who participated in the educational process increased his accuracy from 50% to 90% over the same time period.

Conclusions

When residents finish their surgical training they need to be ready to enter the world of business. These needs will be present whether pursuing a career in academic medicine or the private sector. A program that focuses on the business aspect of surgery enables the residents to better navigate the future while helping to fulfill the systems-based practice competency.     

Minimally invasive training during surgical residency

The field of postgraduate minimally invasive surgery training has undergone substantial growth and change. A survey was sent to all program directors in surgery. Minimally invasive training patterns, facilities, their views, and performance of residents were examined. Ninety-five directors (38%) responded to the questionnaire. Of these, 51 per cent (n = 48) had a program size of three to four residents and 33 per cent (n = 31) had a program size of five to six residents. In 3 per cent of programs (n = 3), residents could not achieve the minimum Accreditation Council for Graduate Medical Education required numbers for advanced laparoscopic cases. Only 47 per cent of programs (n = 45) had dedicated rotations in minimally invasive surgery, ranging from 2 to 11 months. Up to 10 per cent (n = 9) of program directors felt that the current training in minimally invasive surgery was insufficient. Fifty-five per cent (n = 52) felt that laparoscopic adhesiolysis was an advanced laparoscopic procedure, and 33 per cent (n = 31) felt that there should be a separate minimum requirement for each of the commonly performed basic and advanced laparoscopic cases by Accreditation Council for Graduate Medical Education. Fifty-six per cent (n = 53) of programs were performing robotic surgery. Minimally invasive surgery training for surgical residents needs to increase opportunities so that they are able to perform laparoscopic procedures with confidence. There should be specific number requirements in each category of individual basic and advanced laparoscopic procedures.     

A preliminary review of a pilot curriculum to teach open surgical skills during general surgery residency with initial feedback

IntroductionThe aim of this study is to provide a preliminary review of a pilot curriculum to teach open surgical skills during general surgery residency and obtain initial feedback.Data sourceThe general surgery residency program introduced an open surgical skills training curriculum in 2009. The skills sessions are undertaken under the guidance of the faculty. An annual survey was distributed to the residents and faculty to obtain their feedback.ConclusionsA total of 50 sessions were conducted over the last 2 years. Ninety-five percent of the residents perceived this educational activity to be above average to exceptional with nearly 70% rating it as exceptional. Sixty-three percent of the faculty perceived it as above average to exceptional, with nearly 40% rating it as exceptional. The open surgical skills training curriculum was rated as the most educational activity in the program by residents and faculty alike.     

Necessity for improvement in endoscopy training during surgical residency

Background

The Residency Review Committee for Surgery has recently increased the required number of cases needed to achieve competency in endoscopy training.

Methods

A 10-question survey was sent to program directors for general surgery residencies. Endoscopic training patterns, facilities, perspectives, and residents' performance were examined.

Results

Seventy-one surgery programs (30%) responded to the survey. Of these, 42% (n = 30) had a program size of 3 to 4 residents. Ten percent (n = 7) of all programs could not fulfill the minimum Accreditation Council for Graduate Medical Education (ACGME) requirements. Only 55% (n = 39) of programs had a dedicated rotation in endoscopy and an endoscopic skills training laboratory in their program. Few programs had their residents performing more than 100 cases of gastroscopy (18%) and colonoscopy (21%).

Conclusions

Future endoscopy training for surgical residents needs to be improved to comply with the new requirements. This would include provision of an endoscopic skills laboratory, dedicated endoscopic rotations, and increasing the number of staff surgeons who perform endoscopic procedures.     

Patient safety curriculum for surgical residency programs: results of a national consensus conference

BACKGROUND: The American College of Surgeons (ACS) and the Accreditation Council for Graduate Medical Education (ACGME) are committed to promoting patient safety through education. In view of the critical role of residents in the delivery of safe patient care, the ACS and ACGME sponsored jointly a national consensus conference to initiate the development of a curriculum on patient safety that may be used across all surgical residency programs. CONCLUSIONS: National leaders in surgery with expertise in surgical care and surgical education, patient safety experts, medical educators, key stakeholders from national organizations, and surgical residents were invited to participate in the conference. Attendees considered patient safety issues within the context of the 6 core competencies defined by the ACGME and American Board of Medical Specialties (ABMS). Discussions resulted in the development of a curriculum matrix that includes listings of patient safety topics, teaching and learning strategies, and assessment methods. Guidelines for implementation and dissemination are also provided. The curriculum content underscores the need to create an organizational culture of safety and focuses on both individuals and systems. Individual residency programs may prioritize the curriculum content based on their specific needs. The ACS and ACGME will pursue development of educational modules to address the curriculum content, disseminate helpful information, and assist in implementation of new educational interventions. This effort has the potential to positively impact residency education in surgery, help surgical program directors address the core competencies, and enhance patient safety.     

Integrated flexible endoscopy training during surgical residency

Background  New advances in endoscopic surgery make it imperative that future gastrointestinal surgeons obtain adequate endoscopy skills. An evaluation of the 2001–02 general surgery residency endoscopy experience at the University of Missouri revealed that chief residents were graduating with an average of 43 endoscopic cases. This met American Board of Surgery (ABS) and Accreditation Council for Graduate Medical Education (ACGME) requirements but is inadequate preparation for carrying out advanced endoscopic surgery. Our aim was to determine if endoscopy volume could be improved by dedicating specific staff surgeon time to a gastrointestinal diagnostic center at an affiliated Veterans Administration Hospital. Methods  During the academic years 2002–05, two general surgeons who routinely perform endoscopy staffed the gastrointestinal endoscopy center at the Harry S. Truman Hospital two days per week. A minimum of one categorical surgical resident participated during these endoscopy training days while on the Veterans Hospital surgical service. A retrospective observational review of ACGME surgery resident case logs from 2001 to 2005 was conducted to document the changes in resident endoscopy experience. The cases were compiled by postgraduate year (PGY). Results  Resident endoscopy case volume increased 850% from 2001 to 2005. Graduating residents completed an average of 161 endoscopies. Endoscopic experience was attained at all levels of training: 26, 21, 34, 23, and 26 mean endoscopies/year for PGY-1 to PGY-5, respectively. Conclusions  Having specific endoscopy training days at a VA Hospital under the guidance of a dedicated staff surgeon is a successful method to improve surgical resident endoscopy case volume. An integrated endoscopy training curriculum results in early skills acquisition, continued proficiency throughout residency, and is an efficient way to obtain endoscopic skills. In addition, the foundation of flexible endoscopic skill and experience has allowed early integration of surgery residents into research efforts in natural orifice transluminal endoscopic surgery. Presented at the Society of American Gastrointestinal and Endoscopic Surgeons 2006 Annual Meeting, Dallas, Texas, April 2006 (Poster of distinction)     

A cost-effective proficiency-based knot-tying and suturing curriculum for residency programs

BACKGROUND: The purpose of this study was to develop a structured open skills curriculum for knot-tying and suturing using expert-derived performance goals and to examine its feasibility, cost-effectiveness, and construct validity. METHODS: Using commercially available bench models, 11 standardized tasks (ranging from 2-handed knot-tying to running subcuticular closure) were developed and scored using previously validated metrics based on time and errors. Expert performance was used to establish training endpoints and to create a video tutorial. PGY 1 residents (n = 4) were enrolled in a prospective Institutional Review Board-approved pilot study that included proctored orientation and baseline testing, self-training to proficiency, and proctored post-testing (conducted over a 4-wk period). Baseline trainee scores were compared with expert scores to evaluate construct validity. RESULTS: The 11 tasks proved relatively robust, and excellent feedback was obtained from the trainees regarding educational benefit. Overall, trainees performed 144 +/- 33 repetitions over 11 +/- 2 h. Trainees achieved proficiency for 4.6% of the 11 tasks at baseline, 91% during training, and 84% at post-testing. Trainees demonstrated significant improvement from baseline to post-testing, validating skill acquisition; baseline trainee and expert performance were significantly different, confirming construct validity. Curriculum development cost $1200 and required 72 man-hours. Incremental training cost less than $12 per participant and required 8 man-hours per rotation using the video-based self-practice curriculum. In response to participant feedback, two of the 11 tasks were modified and a twelfth task was added. CONCLUSIONS: This curriculum is cost-effective, feasible within the context of residency training, educationally beneficial, and demonstrates construct validity. More widespread adoption of standardized, validated skills curricula such as this by residency programs is warranted.     

'Blueprint' for designing a curriculum for intern and residency programs.

Postgraduate training faces new challenges as physicians become more specialized, the lay population becomes more sophisticated, and those resources necessary to train physicians become scarcer. A nine-step model is presented for developing a curriculum that will prepare healthcare professionals to practice in this changing socioeconomic milieu.     

The professionalism curriculum as a cultural change agent in surgical residency education

BackgroundTeaching professionalism effectively to fully engaged residents is a significant challenge. A key question is whether the integration of professionalism into residency education leads to a change in resident culture.MethodsThe goal of this study was to assess whether professionalism has taken root in the surgical resident culture 3 years after implementing our professionalism curriculum. Evidence was derived from 3 studies: (1) annual self-assessments of the residents' perceived professionalism abilities to perform 20 defined tasks representing core Accrediting Council on Graduate Medical Education professionalism domains, (2) objective metrics of their demonstrated professionalism skills as rated by standardized patients annually using the objective structure clinical examination tool, and (3) a national survey of the Surgical Professionalism and Interpersonal Communications Education Study Group.ResultsStudy 1: aggregate perceived professionalism among surgical residents shows a statistically significant positive trend over time (P = .016). Improvements were seen in all 6 domains: accountability, ethics, altruism, excellence, patient sensitivity, and respect. Study 2: the cohort of residents followed up over 3 years showed a marked improvement in their professionalism skills as rated by standardized patients using the objective structure clinical examination tool. Study 3: 41 members of the national Surgical Professionalism and Interpersonal Communications Education Study Group rated their residents' skills in admitting mistakes, delivering bad news, communication, interdisciplinary respect, cultural competence, and handling stress. Twenty-nine of the 41 responses rated their residents as “slightly better” or “much better” compared with 5 years ago (P = .001). Thirty-four of the 41 programs characterized their department's leadership view toward professionalism as “much better” compared with 5 years ago.ConclusionsAll 3 assessment methods suggest that residents feel increasingly prepared to effectively deal with the professionalism challenges they face. Although professionalism seminars may have seemed like an oddity several years ago, residents today recognize their importance and value their professionalism skills. As importantly, department chairpersons report that formal professionalism education for residents is viewed more favorably compared with 5 years ago.     

A multidisciplinary systems-based practice learning experience and its impact on surgical residency education

OBJECTIVE: To design and implement a multidisciplinary systems-based practice learning experience that is focused on improving and standardizing the preoperative quality of care for general surgical patients. DESIGN: Four parameters of preoperative care were designated as quality assessment variables, including bowel preparation, perioperative beta-blockade, prophylactic antibiotic use, and deep venous thrombosis prevention. Four groups of general surgery residents (PGY I-V), each led by 1 chief resident, were assigned a quality parameter, performed an evidence-based current literature review, and formulated a standardized management approach based on the level of evidence and recommendations available. Because preoperative preparation includes anesthetic care and operating room preparation, we presented our findings at the Department of Surgery Grand Rounds in a multidisciplinary format that included presentations by each resident group, the Department of Anesthesia, the Department of Medicine, and the Department of Nursing. The aim of the multidisciplinary quality assurance conference was to present the evidence-based literature findings in order to determine how standardization of preoperative care would alter anesthetic and nursing care, and to obtain feedback about management protocols. To determine the educational impact of this model of integrated systems-based practice quality assessment on the teaching experience, residents were queried regarding the value of this educational venue and responses were rated on a Likert scale. RESULTS: Resident participation was excellent. The residents garnered valuable information by performing a literature review and evaluating the best preoperative preparation given each parameter. Furthermore, integration of their findings into systems-based practice including anesthesia and nursing care provided an appreciation of the complexities of care as well as the associated need for appropriate medical knowledge, communication, and professionalism. The derivation of treatment protocols included an opportunity to incorporate several competencies across multiple disciplines. The residents evaluated 5 questions and deemed the educational exercise an effective model to enrich surgical resident education while simultaneously improving patient care. The residents also strongly agreed that they would participate in similar projects in the future as well as recommend this educational exercise to other residents. A finalized preoperative order set was created and distributed to all residents for use in the preoperative care of general surgery patients. CONCLUSIONS: Our multidisciplinary systems-based practice learning experience focused on improving and standardizing the preoperative quality of care for patients, and general surgery residents were pivotal participants in that process. This exercise had a positive impact on our general surgery residency education program and proved to be a valuable model of systems-based practice competency.     

A new residency program for Japanese surgical specialties

Medical education, in general, should be a continuous and life-long persistent training for the purposes to be reached through the education of humanity and medical techniques which are the arts based on the sciences whichever originated from the western or eastern world histories. These training started from the student life through the termination of medical profession, especially in the field of surgery, the medical doctors are the only existence to be allowed to insult human body even for the purpose of treatment. These facts should be learned through well-trained experienced teachers with ethics, theories and evidence based medicine. The new training curriculum for 5 years program of the surgical specialties in Japan has built up and started just two years ago to be successful supported not only by medical teachers but also by the government and civilian economical foundations. A sort of national board of medical specialties is necessary system to develop general medical care system with effective specializations.