The initial examination for a Certificate of Added Qualifications in Surgery of the Hand

A total of 510 candidates took the 1989 Examination for Added Qualifications in Surgery of the Hand, including 412 diplomates of the American Board of Orthopaedic Surgery and ninety-eight diplomates of the American Board of Surgery. Most candidates reported that they had intensive practices in hand surgery and large annual case-loads, and most had taken a hand fellowship. However, there were significant differences between diplomates of the American Board of Orthopaedic Surgery and diplomates of the American Board of Surgery regarding these variables. The psychometric characteristics of the examination were very good. The average difficulty value was 77.6 per cent correct. The average item-discrimination value was high, and the total test reliability coefficient was 0.89. In general, the candidates' performance was very good, although there was a wide range in scores. A passing score of 66.3 per cent correct was selected, resulting in an over-all failure rate of 7.6 per cent, with 471 candidates passing and thirty-nine failing the examination. There were significant relationships between performance on the examination and several background variables, such as percentage of practice in hand surgery, having taken a hand fellowship, and size of the annual case-load.     

Maintaining outcomes in a surgical residency while complying with resident working hour regulations

BACKGROUND: Regulatory requirements for resident working hours were designed to improve patient care. Compliance challenges a training program to meet procedural and clinical requirements. This is a retrospective study of a 5-year experience in addressing the challenges and studying the impact of compliance on resident caseload and board performance. METHODS: Our surgical program adopted strict start/stop working hours for clinical contact. Program leadership modified the program to establish procedural and performance criteria. Procedures were prioritized and assignments were changed to maximize clinical and procedural experience while reducing redundancy of experience. Procedural activity was monitored frequently. Compliance with working hour regulations was monitored and behavior modified where necessary. A web based computer program was developed to improve measurement of compliance and provide feedback. Outcome measures included both the number of procedures as reported by the ACGME and performance on the American Board of Surgery, Qualifying Examination. RESULTS: Working hour compliance is greater than 95%. First time pass rate on the Qualifying examination is 90% (45/50). There is no significant difference in the procedural activity. CONCLUSION: Complying with working hour regulations improves the quality of a resident's life and can be achieved while maintaining procedural experience and guaranteeing academic development.     

Influence of programmed textbook review on American Board of Surgery In-service Examination scores

Documentation of academic achievement throughout training is an essential component to residency accreditation. Resident performance evaluations by attending staff provide good assessments of affective behavior and technical skills, but evaluation of surgical knowledge is often subjective in nature. The American Board of Surgery In-Service Examination has become the standard by which academic performance is measured. Numerous types of educational programs are used to increase the knowledge base of residents. Sporadic reading of a standard surgical text is a common part of many residency programs. In 1981, the Michigan State University/Butterworth Hospital General Surgery Residency opted to drop the surgical text reading in favor of a review of specific topics attuned to resident needs. Results of the 1982 American Board of Surgery (ABS) examination identified a significant decrease in percentile scores and average resident performance. A program requiring a systematic review of a standard surgical text was then initiated. This review involved assigned reading and administration of weekly examinations covering each assignment. Following institution of this program, a dramatic improvement in the total average scores for the entire resident group was noted. Similar improvement was noted in individual group scores (PG I, II, III, IV, and V). Results of this study indicate that reading of a standard surgical text should be a required part of each residency program.     

Predictors of Success on the American Board of Orthopaedic Surgery Examination

Predictors of success of orthopaedic residents on the American Board of Orthopaedic Surgery (ABOS) examination are controversial. We therefore evaluated numerous variables that may suggest or predict candidate performance on the ABOS examination. We reviewed files of 161 residents (all graduates) from one residency program distributed into two study groups based on whether they passed or failed their first attempt on the ABOS Part I or Part II examination from 1991 through 2005. Predictors of success/failure on the ABOS I included the mean percentile score on the Orthopaedic In-Training Examination (OITE) (Years 2 through 4), the percentile OITE score in the last year of training, US Medical Licensing Examination (USMLE) score, Dean’s letter, election to Alpha Omega Alpha (AOA), and number of honors in selected third-year clerkships. All but the USMLE score predicted passing the ABOS Part II examination. These data suggest there are objective predictors of residents’ performance on the ABOS Part I and Part II examinations. Each author certifies that he has no commercial associations (eg, consultancies, stock ownership, equity interest, patient/licensing arrangements, etc) that might pose a conflict of interest in connection with the submitted article.     

Do Scores of the USMLE Step 1 and OITE Correlate with the ABOS Part I Certifying Examination?: A Multicenter Study

Background  

The US Medical Licensing Examination (USMLE) and Orthopaedic In-Training Examination (OITE) are commonly used to select medical students or residents, respectively. Knowing how well these examinations predict performance on the American Board of Orthopaedic Surgery (ABOS) Part I certifying examination is important to provide evaluations for medical students and residents. Previous studies comparing the OITE scores with the ABOS Part 1 scores have been limited to one program.     

Factors affecting performance on the American Board of Surgery in-training examination.

BACKGROUND: Surgical programs use the American Board of Surgery In-Training Examination (ABSITE) in several ways, including for promotion and retention decisions. The purpose of this study was to identify the combination of factors that explain residents' successful performance on the ABSITE. METHODS: Fifty-one surgical residents completed questionnaires, and information was gathered about their previous ABSITE performance, anxiety, probationary status, amount of study, amount of sleep before examination, confidence, and attendance at conference. RESULTS: An analysis of the data for those with experience taking the examination (n = 34) indicated that the combination of conference attendance (26.3%), previous performance (16.5%), probationary status (10.4%), amount of sleep (9.8%), and amount of study (8%) were significant in explaining a total of 71% of the variance in ABSITE scores. Amount of study (+0.32, P = 0.011), confidence (+0.36, P = 0.005), and conference attendance (+0.51, P = 0.001) were significantly correlated with ABSITE performance. CONCLUSIONS: Several factors contribute to residents' successful ABSITE performance. These findings may lead to improved examination performance and application of knowledge, both during residency and throughout their career.     

Preliminary assessment of a scientific curriculum in a surgical residency program.

A structured, basic science curriculum was instituted for surgical residents of the University of Connecticut (Farmington) Integrated Residency Program during the 1990-1991 academic year in concordance with American Board of Surgery guidelines. The impact of the new program was measured by comparing performance on monthly basic science examinations, the in-training examination, and "mock" oral examinations for the 1990-1991 academic year with that of the preceding academic year. While monthly examination scores improved for the entire group of residents (67.7 vs 64.6), in-training and oral examination scores did not change significantly. Categorical residents generally demonstrated superior performance and greater improvement than did preliminary residents. Data analysis suggested that the new curriculum was an effective educational device and that university-designed monthly examinations were valid testing instruments, but there was an apparent incongruity between the goals of the curriculum and the American Board of Surgery In-Training Examination.     

Effect of a cardiac-specific didactic course on thoracic surgery in-training examination performance

BACKGROUND: The purpose of this study was to determine which factors influenced performance on the Thoracic Surgery In-Training Examination (TSITE) and whether the addition of a cardiac-specific didactic study course improved scores. METHODS: Between 1989 and 2002, 59 TSITE scores (overall, cardiac [C-TSITE], and thoracic [T-TSITE]) were collected from 33 residents (23 cardiac track, 10 thoracic). Factors assessed with univariate and multivariate analysis included calendar year, year of training (31 year I, 28 year II), standardized test-taking history (average National Board and American Board of Surgery in-training scores), subjective faculty assessment of cardiothoracic knowledge, months on cardiac versus thoracic service, clinical performance, and participation in a cardiac-specific didactic series with faculty lectures and board question reviews (12 residents). RESULTS: Cardiac-track residents had higher C-TSITE percentile scores (53% +/- 27% versus 38% +/- 27%, p < 0.05), whereas thoracic-track residents had higher T-TSITE scores (70% +/- 24% versus 51% +/- 25%, p < 0.01). Multivariate analysis identified 3 factors associated with higher overall TSITE scores: standardized test-taking history (p < 0.001), subjective faculty assessment of knowledge (p < 0.001), and year of training (p < 0.007). Inclusion in the cardiac-specific didactic series did not affect C-TSITE scores (50% +/- 30% versus 48% +/- 27%, p > 0.82) or overall TSITE scores (p > 0.23). CONCLUSIONS: Standardized test-taking history and subjective faculty assessment of knowledge were associated with higher TSITE scores, but implementation of a cardiac-specific didactic series had no influence. These findings suggest that independent study and reading may be the best way to improve TSITE scores.     

Effect of a resident self-study and presentation program on performance on the thoracic surgery in-training examination.

BACKGROUND: In this study we examine whether conversion from a didactic lecture format to a resident self-study and presentation program can improve performance on the Thoracic Surgery In-Training Examination (TSITE). METHODS: During the first 5 years, educational conferences were didactic lectures delivered by the attending thoracic surgery staff (group 1, n = 9 residents). During the second 5 years, residents prepared and delivered reviews from major textbook sources (group 2, n = 9 residents). Scores on the American Board of Surgery In-Training Examination (ABSITE) as a chief resident in general surgery were analyzed using one-way analysis of variance to assess fund of knowledge and test-taking skills prior to thoracic surgery training for the two groups. Scores on the TSITE during the first and second years of thoracic surgery training were recorded for each resident and analyzed using a paired t test. The data are expressed as the mean +/- standard deviation. RESULTS: Eighteen thoracic surgery residents over a 10-year period were involved in the study. ABSITE scores as a chief resident in general surgery did not differ between the two groups. Residents in group 1 improved their percentile rank from the first to the second year by a mean of 11%+/-12%, whereas those in group 2 improved their scores by a mean of 31%+/-21% (P < 0.05). CONCLUSIONS: When compared with a didactic lecture format, a resident self study and presentation program improves performance on the Thoracic Surgery In-Training Examination. This improvement in performance typically manifests during the second year of thoracic surgery training.     

Correlating selection criteria with subsequent performance as residents

The objective of this study was to determine which criteria in the residency application had the highest correlation with subsequent performance of orthopaedic residents. Data collected from the application files of 58 residents included scores on standardized tests, number of honors grades in the basic and clinical years of medical school, election to Alpha Omega Alpha, numbers of research projects and publications, and numbers of extracurricular activities. Measures of performance included scores on the Orthopaedic In-Training Examination and American Board of Orthopaedic Surgery Part I Examination, and faculty evaluations of overall, cognitive, affective, and psychomotor performance. The number of honors grades on clinical rotations was the strongest predictor of performance, whereas election to Alpha Omega Alpha was second. The only other significant correlation was between the number of fine motor activities and psychomotor performance. None of the predictor variables had a significant correlation with Orthopaedic In-Training Examination or American Board of Orthopaedic Surgery Examination scores. Consistency between faculty rankings in each of the four categories was supported by regression analysis. From the results of this study, it appears that academic performance in clinical clerkships in medical school is the most predictive of resident performance. Range restriction in the data available for orthopaedic residency applicants, however, likely precludes the development of a reliable model to assist in the selection of orthopaedic residents.     

Weekly reading assignments and examinations result in sustained improvement in American Board of Surgery In-Training Examination (ABSITE) scores

The American Board of Surgery In-Training Examination (ABSITE) score is used by general surgery training programs as a measure of the medical knowledge component of the Accreditation Council of Graduate Medical Education competencies. Poor performance on ABSITE (< 35%) has correlated with high failure rates on the American Board of Surgery Qualifying Examination. We previously demonstrated an improvement in ABSITE scores over a 1-year period through the initiation of weekly reading assignments and review examinations. We sought to determine whether this educational endeavor would result in a sustained improvement in ABSITE scores over several years. ABSITE scores from two successive, 3-year time periods (1999-2001, Group 1, 2002-2004, Group 2), were compared. For Group 1, no formal reading assignments or preparatory exams were given. For Group 2, weekly reading assignments and examinations were administered by the faculty. For Group 1, the mean ABSITE score was 60 per cent, versus 75 per cent (P < 0.01) for Group 2. In Group 1, 13 out of 76 scores (17%) were below 35 per cent, whereas in Group 2, only 4 out of 74 scores (5%) were below 35 per cent (P = 0.03). Sustained improvement in ABSITE scores and fewer scores < 35 per cent were achieved over a 3-year period with an educational program of weekly topic-specific reading assignments and written review examinations.     

American Board of Surgery examinations: can we identify surgery residency applicants and residents who will pass the examinations on the first attempt?

Background

The Residency Review Committee requires that 65% of general surgery residents pass the American Board of Surgery qualifying and certifying examinations on the first attempt. The aim of this study was to identify predictors of successful first-attempt completion of the examinations.

Methods

Age, sex, Alpha Omega Alpha Honor Medical Society status, class rank, honors in third-year surgery clerkship, interview score, rank list number, National Board of Medical Examiners/United States Medical Licensing Examination scores, American Board of Surgery In-Training Examination scores, resident awards, and faculty evaluations of senior residents were reviewed. Graduates who passed both examinations on the first attempt were compared with those who failed either examination on the first attempt.

Results

No subjective evaluations of performance predicted success other than resident awards. Significant objective predictors of successful first-attempt completion of the examinations were Alpha Omega Alpha status, ranking within the top one third of one's medical student class, National Board of Medical Examiners/United States Medical Licensing Examination Step 1 (>200, top 50%) and Step 2 (>186.5, top 3 quartiles) scores, and American Board of Surgery In-Training Examination scores >50th percentile (postgraduate years 1 and 3) and >33rd percentile (postgraduate years 4 and 5).

Conclusions

Residency programs can use this information in selecting residents and in identifying residents who may need remediation.     

AOA Continuing Medical Education

The author provides an update on the current CME cycle, which began on January 1, 2010, and will end on December 31, 2012. The author also details minor changes to the requirements for Category 1 CME sponsors accredited by the AOA and describes new online CME opportunities. The current article also explains changes regarding the AOA's awarding and recording of specialty CME credit hours for AOA board-certified osteopathic physicians. In addition, the article includes information to assist osteopathic specialists and subspecialists in requesting AOA Category 1-A credit for courses accredited by the Accreditation Council for Continuing Medical Education.     

Achieving educational excellence: a strategic initiative to enhance orthopedic resident academic performance

Orthopedic resident training involves not only the hands-on learning of surgery but also should equally involve instructing the core knowledge of musculoskeletal medicine. Our program has developed a strategy that enhances resident educational performance; the educational curriculum entails conferences daily. Conferences include gross and surgical anatomy, orthopedic basic science, multidisciplinary trauma, radiology, pathology, journal club, and orthopedic subspecialty conferences. The primary purpose of the conference schedule is to provide the residents with a comprehensive education in orthopedic surgery. It is not geared toward taking the Orthopaedic In-Training Examination (OITE). The OITE is administered annually by the American Academy of Orthopaedic Surgeons (AAOS) and serves as an objective measure of knowledge acquisition. There has been a scientifically validated correlation between performance on the OITE and passage of the American Board for Orthopaedic Surgery Part I Examination. As a collective program, we have achieved at or above the 98th percentile nationally from 2004 to 2009. This academic success has not impacted the total surgical case volume negatively or interfered with Residency Review Committee (RRC) policies.     

Administration and performance on the Spinal Cord Injury Medicine Certification Examination over a 10-year period

Context/Objective: The examination for Spinal Cord Injury (SCI) Medicine subspecialty certification has been administered since 1998, but published information about exam performance or administration is limited.

Design: Retrospective review

Setting/Participants: We examined de-identified information from the American Board of Physical Medicine and Rehabilitation (ABPMR) database for characteristics and performance of candidates (n?=?566) who completed the SCI Medicine Examination over a 10-year period (2005–2014), during which the exam outline and passing standard remained consistent.

Interventions: Not applicable

Outcome Measures: We analysed candidate performance by candidate track, primary specialty, number of attempts, and domains being tested. We also examined candidate perception of the SCI Medicine Exam by analysing responses to a survey taken after exam completion.

Results: Thirty-six percent of candidates who completed the exam during the study period took it for initial certification (23% in the fellowship track and 13% in the practice track offered during the initial “grandfathering” period) and 64% took it for maintenance of certification (MOC) in SCI Medicine. Factors associated with better exam performance included primary specialty certification in Physical Medicine and Rehabilitation (PM&R) and first attempt at passing the exam. For PM&R candidates, ABPMR Part I Examination scores and SCI Medicine Examination scores were strongly correlated. Candidate feedback about the exam was largely positive with 97% agreeing or strongly agreeing that it was relevant to the field and 90% that it was a good test of their knowledge.

Conclusion: This study can inform prospective candidates for the SCI Medicine Examination as well as those guiding them. It may also provide useful information for future exam development.     

Assessment of medical student fund of knowledge in surgery

A medical student's fund of knowledge can be assessed by either his demonstrated fund of knowledge on a clinical service or an examination at the completion of the clerkship. During the past 2 years we have evaluated each student's fund of knowledge in two ways: clinical assessment by faculty and residents and performance on the National Board of Medical Examiners Part II Special Surgical Examination given at the end of the surgical clerkship. This study compares the clinical assessment of surgical knowledge for all 100 members of the class of 1984 at the University of Utah with other subjective evaluations such as attitudes and college grade point average and objective measures of performance such as premed MCAT score, Part I National Boards score, and the National Boards Special Surgical Examination. The correlation between clinical assessment of knowledge and the Surgical Examination was 0.23, with a range on individual services from -0.42 to 0.45. There was a higher correlation for the total group of 0.56 between fund of knowledge and attitudes, with a range on individual services of -0.04 to 0.72. The correlations between clinical assessment of knowledge and Part I National Boards, college grade point average, and chemistry MCAT score were 0.23, 0.09, and 0.15, respectively. These results indicate that the clinical assessment of fund of knowledge is not a good predictor of performance on the surgical section of the National Boards. Clinical assessment of fund of knowledge appears to be linked more closely to faculty and resident assessment of student attitudes. There are a couple possible explanations for these results: clinicians are measuring different aspects of knowledge than are National Boards or clinicians do not accurately assess knowledge and confuse attitudes such as interest and enthusiasm with fund of knowledge.     

The role of the community teaching hospital in surgical undergraduate education

BACKGROUND: Significant changes in surgical practice have resulted in a reexamination of surgical undergraduate education. The increasing emphasis toward ambulatory procedures positions the community hospital as an excellent alternative site for surgical education. This study compares the quality of one medical school's surgical education at a principal teaching hospital to that of affiliated teaching hospitals. METHODS: Surgical undergraduate education offered through four programs was evaluated for 1993 to 1997. Students' performance was objectively rated by the National Board Examination in surgery, an oral examination, and a clinical appraisal. A subjective appraisal was determined via students' clerkship evaluation. RESULTS: There was a significant difference (P <0.01) in National Board Examination scores and clerkship evaluations that favored some affiliated teaching hospitals over the principal teaching hospital. CONCLUSION: The quality of surgical undergraduate education, documented by objective testing and subjective perception, indicated that the education obtained at the affiliated hospitals was at least equivalent to the principal teaching hospital.     

AOA continuing medical education

The current continuing medical education (CME) cycle began on January 1, 2007, and will end on December 31, 2009. Final statistics for the 2004-2006 CME cycle will not be available until May 31, 2008. The author provides an update on trends in osteopathic CME programs, details minor changes to the requirements for AOA-accredited Category 1 CME sponsors, and describes new online CME opportunities. In addition, the current article explains changes regarding the American Osteopathic Association's awarding and recording of CME credit hours for physicians who have specialty board certification.     

Maintenance of certification: How to stay board certified by the American board of colon and rectal surgery

Achieving board certification is a milestone in the life of a young surgeon. The American Board of Colon and Rectal Surgery (ABCRS) Certifying Examination is considered the penultimate test of professional achievement for those who have completed an Accreditation Council for Graduate Medical Education (ACGME) sanctioned general surgery and colorectal residencies followed by certification by the American Board of Surgery. The mission of the American Board of Colon and Rectal Surgery, incorporated as the American Board of Proctology in 1935, contemplates the establishment of standards and norms of knowledge by which physicians in the field of colon and rectal surgery are specifically measured to ensure the safety of the American public. These standards and requirements have changed over the long life of the American Board of Colon and Rectal Surgery, and staying current with requirements not only for achieving ABCRS certification, but for maintaining certification is important. The term “Maintenance of Certification” has recently been replaced by “Continuous Certification” and refers to a system of continuous assessment of common knowledge that every board certified colorectal surgeon should understand. In addition to demonstrating factual knowledge, a Board Certified colon and rectal surgeon should be able to demonstrate evidence of professionalism, provide evidence of commitment to lifelong learning, and demonstrate participation in activities that result in “practiced based improvement.” The purpose of this chapter is to specifically inform board certified colon and rectal surgeons of the procedures in place to stay certified. A brief history of board certification and data from other specialties on the impact of continuing certification is useful for context and provided.     

The impact of work hour restrictions on surgical resident education

OBJECTIVE: Resident work-hour restrictions were instituted in July 2003 based on ACGME mandates. The American Board of Surgery In-Training Examination (ABSITE), American Board of Surgery Qualifying Examination (ABSQE), and operative volume traditionally have been measures of surgical resident education and competency. The objective of this study was to determine the effect of reduced work hours on resident standardized test scores and operative volume at our institution. DESIGN: We reviewed ABSITE scores, ABSQE scores, and operative logs from 1997 to 2005 of all general surgery residents. Linear mixed-effects models were fitted for each component ABSITE score (total, basic science, and clinical management), and they were compared using a chi-squared likelihood ratio. Operative logs of graduating residents were compared before and after the work restrictions and were evaluated for association with ABSITE score. p-values less than 0.05 were considered significant. RESULTS: The program was compliant with ACGME mandates within 6 months of institution. ABSITE scores improved significantly after the restriction of work hours in both basic science (p = 0.003) and total score (p = 0.008). Clinical management scores were not affected. The number of major cases recorded by graduating residents did not change. A positive correlation was found between number of cases performed during residency and clinical management ABSITE scores (p = 0.045). ABSQE scores were not impacted by operative volume during residency. CONCLUSIONS: ABSITE scores improved significantly after the restriction of resident work hours. Resident operative experience was not affected. An unexpected consequence of work-hour restrictions may be an improvement in surgical resident education.