Evaluation of clinical experience in a radiology residency program with quantitative profiling: rationale,methods, and application

RATIONALE AND OBJECTIVES: The author developed a technique for residency program evaluation, called "quantitative profiling," that is based on computer retrieval of radiologic reports. The hypothesis was that it would provide insights into the contributions of residents to clinical service, measures of resident experience and productivity for program evaluation, and benchmarks for comparison. MATERIALS AND METHODS: The radiology residency program of a major teaching hospital was studied retrospectively from 1989 to 1997. The number of radiologic reports signed by individual residents and faculty members was retrieved. The clinical experience of the 1993-1997 cohort of residents was described according to subspecialty area and modality. RESULTS: Residents signed 46.5% of all reports, with a mean total of 14,445 reports +/- 1,292 per resident during the entire residency. The distribution of examinations was as follows: thoracic, 42.1%; musculoskeletal, 26.1%; abdominal, 13.4%; sonography, 8.7%; neuroradiology, 4.3%; nuclear, 2.4%; breast, 1.6%; and vascular, 1.4%. The most frequently reported results were for one-view chest radiography. CONCLUSION: Quantitative profiling can help track the range and progress of resident experience, help determine the deployment of residents, and provide empirical data upon which decisions to modify residency programs may be based.     

Misrepresentation of publications by radiology residency applicants

RATIONALE AND OBJECTIVES: The authors' purpose was to determine the extent of misrepresentation of research publications by radiology resident applicants. MATERIALS AND METHODS: The authors reviewed 379 consecutive applications, including curricula vitae, for a radiology residency program in 1996. All reported publications and "in-press" articles were checked by means of a MEDLINE search. RESULTS: Of the 379 applicants, 108 were from medical schools in the United States, and 271 were from international medical schools. Seventy-three applicants listed articles published or in press on their applications (24 U.S., 49 international applicants). Of 286 separate citations in the applications, 105 were found with the MEDLINE search, and 181 were not found. Of the latter, 168 cited journals were not indexed in MEDLINE or the applicants did not include sufficient information to verify their existence. Thirteen citations (from eight applicants; three U.S., five international) were not found even though they cited journals indexed by MEDLINE. CONCLUSION: Of all applicants reporting publications, 11% likely misrepresented them on their applications. A large percentage of citations, however, could not be verified because of insufficient information in the citation or claimed publication in a journal not available on MEDLINE. Radiology residency program directors should be aware of this uncommon, but important, problem.     

Diagnostic radiology resident compliance with recommended health guidelines: effect of resident work environment

RATIONALE AND OBJECTIVES: To determine diagnostic radiology resident compliance with recommended health guidelines for physical activity, body weight, diet, related health indicators, and the effects of the resident work environment on compliance. MATERIALS AND METHODS: A request was electronically mailed to members of the Association of Program Directors in Radiology and the Association of Program Coordinators in Radiology in May 2007 and again in June 2007, asking members to forward to their radiology residents an invitation to complete an online health survey. Frequency counts and Fisher's exact test, respectively, were used to summarize results and to determine statistically significant relationships between survey variables. RESULTS: A total of 811 radiology residents completed the survey, representing 18% of 4,412 diagnostic radiology residents. Five hundred forty-five (67.2%) of 811 were male and 264 (32.6%) female. Two hundred ten (25.9%) were first-year, 239 (29.5%) second-year, 201 (24.8%) third-year, and 161 (19.9%) fourth-year residents. Three hundred two (37.2%) engaged in recommended guidelines for physical activity and < or =465 (57.3%) complied with each of multiple federal dietary guidelines (excluding alcohol intake). Up to 329 (40.6%) residents did not know whether they were in compliance with various dietary guidelines. A total of 426 (52.5%) residents reported working > or =60 hours/week, which significantly correlated with less physical activity (P = .013). CONCLUSION: A substantial number of residents are out of compliance with federal guidelines for physical activity and diet and are not knowledgeable about their personal dietary intake. Long work hours are related to a lack of physical activity. Radiology programs may be able to influence resident health practices by modifying work hours and the working environment, encouraging healthy dietary intake and physical activity, and instituting campaigns to inform residents and faculty about health guidelines and available wellness programs.     

Occupational necessity and educational invention: resident teaching of radiologic technologists

RATIONALE AND OBJECTIVE: Radiology faces a severe and growing shortage of radiologic technologists. One way of redressing this problem is to improve the quality of education provided to radiologic technology students. Yet growing clinical demands increasingly erode faculty time for teaching. This study examined whether radiology residents could provide equivalent instruction in radiologic technology at lower cost, and whether such experience could enhance residents' interest in teaching as part of their careers. MATERIALS AND METHODS: Course evaluation forms completed by the students in a required radiologic pathology course were reviewed, and student-reported faculty and resident performances in teaching were compared. Residents also were surveyed for their reactions to the experience of teaching this course. RESULTS: Ninety percent of students (27 of 30) either agreed or strongly agreed that the course was well taught by radiology faculty members, and 97% (29 of 30) either agreed or strongly agreed that the course was well taught by radiology residents. The total direct cost of instruction by radiology residents was 73% lower than the cost of instruction by faculty. Residents who participated in teaching found the experience worthwhile, and they described a wide variety of personal and educational benefits. CONCLUSION: Involving radiology residents in teaching can help redress the growing shortage of radiologic technologists, relieve some of the pressure on faculty time, and contribute to the professional development of the next generation of radiologists.     

Attitudes of and influences on residents in English Canadian radiology programs regarding interventional radiology: results of a national survey by the Canadian Interventional Radiology Association (CIRA)

PURPOSE: There has been a North American trend toward reduced application to the subspecialty of Interventional Radiology (IR). Out of fear of a looming manpower shortage, this survey was conducted to better understand awareness and attitudes toward IR by radiology residents-in-training. MATERIALS AND METHODS: An anonymous online survey was emailed to the Diagnostic Radiology Residency Program Director/Department Chairperson of each of the 13 English medical schools in Canada, to be forwarded to each respective Radiology Residency Program's radiology residents. The survey was open for a period of 1 month. The survey consisted of 29 questions, which could be answered online using a web-based program. Responses to questions were tabulated and comments recorded. RESULTS: A total of 84 survey responses were received of a possible 333 (25%), including responses from each of the 13 English Programs. Responses regarding demographics, training, careers aspirations and motivations, and influences were collected. Fifty-one percent of respondents reported being either "moderately" or "very" interested in the field of IR; however, only 13% reported intention to perform an IR fellowship. A number of issues were identified as dissuading current radiology residents from pursuing IR, including income, work hours and hours of on-call, and turf issues. CONCLUSION: A number of issues were identified as factors which prevented residents with an interest in IR from applying to IR fellowships. These must be addressed to increase IR recruitment rates of radiology residents.     

Utility of an informational intranet in a radiology department

RATIONALE AND OBJECTIVES: The authors performed this study to assess the usefulness of the various features of their radiology department intranet and to quantify its effect in facilitating intradepartmental information flow. MATERIALS AND METHODS: The number, length, and nature of visits to each directory on the intranet were recorded by a commercially available intranet analysis software package. RESULTS: During the last 6 months of 2001, 89,411 page views (mean, 485 per day) occurred during 26,344 visits to the radiology department intranet. The most commonly accessed features were billing code lists (29% of total views); the intranet home page (27%); faculty telephone and pager numbers (2.6%); digital dictation access codes (1.6%); and magnetic resonance imaging protocols (1.5%). A total of 32,408 files, including daily clinical assignments, monthly fellow and resident schedules, and protocol forms, were downloaded as portable document files by intranet users. CONCLUSION: The intranet has proved an invaluable addition to the authors' departmental infrastructure, as evidenced by its extensive use by the staff. It has facilitated the rapid dissemination of information without hard copy.     

RadioGraphics: a Web-based model for radiology resident self-education

RATIONALE AND OBJECTIVES: The residency review committee (RRC) for diagnostic radiology of the Accreditation Council for Graduate Medical Education mandates core competencies including computer-aided applications in medicine. The purpose of this review was to evaluate the use of RadioGraphics' on-line CME to satisfy the RRC requirements. MATERIALS AND METHODS: Twenty radiology residents at a university training program read the same four articles in the on-line version of RadioGraphics. Before reading each article, the residents took the associated CME pre-test and, after completing the article, the CME post-test. Each resident completed a survey to evaluate the quality of the resident experience using RadioGraphics' CME on-line program after completing the four articles and tests. RESULTS: The combined mean scores of all four articles pre-test and post-test scores were 5.6 and 9.3. Significant improvement in the test scores was determined by a student t-test (P < .001). Fourteen residents agreed and one resident disagreed with the statement that the modules were time effective. Nineteen of 20 residents agreed with the statement that valuable information for future practice was gained and that they would continue to use RadioGraphics for CME in the future. All of the residents agreed with the statement that the experience satisfied the residency's requirement to teach computer skills appropriate for ongoing learning. CONCLUSION: RadioGraphics' on-line CME an effective method to teach residents skills required by the RRC.     

Radiology resident evaluation of head CT scan orders in the emergency department

BACKGROUND AND PURPOSE: Prior studies have revealed little difference in residents' abilities to interpret cranial CT scans. The purpose of this study was to assess the performance of radiology residents at different levels of training in the interpretation of emergency head CT images. METHODS: Radiology residents prospectively interpreted 1324 consecutive head CT scans ordered in the emergency department at the University of Arizona Health Science Center. The residents completed a preliminary interpretation form that included their interpretation and confidence in that interpretation. One of five neuroradiologists with a Certificate of Added Qualification subsequently interpreted the images and classified their assessment of the residents' interpretations as follows: "agree," "disagree-insignificant," or "disagree-significant." The data were analyzed by using analysis-of-variance or chi-squared methods. RESULTS: Overall, the agreement rate was 91%; the insignificant disagreement rate, 7%; and the significant disagreement rate, 2%. The level of training had a significant (P =.032) effect on the rate of agreement; upper-level residents had higher rates of agreement than those of more junior residents. There were 62 false-negative findings. The most commonly missed findings were fractures (n = 18) and chronic ischemic foci (n = 12). The most common false-positive interpretations involved 10 suspected intracranial hemorrhages and suspected fractures. CONCLUSION: The level of resident training has a significant effect on the rate of disagreement between the preliminary interpretations of emergency cranial CT scans by residents and the final interpretations by neuroradiologists. Efforts to reduce residents' errors should focus on the identification of fractures and signs of chronic ischemic change.     

Use of personal digital assistants in diagnostic radiology resident education

RATIONALE AND OBJECTIVES: Personal digital assistants (PDAs) are gaining widespread use in the medical community. We introduced a PDA-based mobile system that provides departmental and educational information with a seamless connection to the intranet. The objective of this study is to determine the impact a PDA has on educational resources (learning or data reference) brought to work and used at home by a radiology resident based on user surveys. MATERIALS AND METHODS: Survey was performed on 32 radiology residents in our department before and 6 months after the release of the PDA-based system. We assessed the changes in (1) sources of learning at home and at work, and in (2) data reference. The second survey also evaluated the usefulness of each component of the system. RESULTS: After the release of the PDA-based mobile system, the use of "digital books and references" as data references and educational resources that were brought to work every day significantly increased (P = .016, P < .0001, respectively). "Traditional books and references" remained the "most useful source in learning radiology"; however, "digital books and references" increased as the residents' first choice from 0% to 16% within 6 months of introducing the package (P = .125). CONCLUSION: The introduction of a PDA-based system consisting of educational and departmental information had a statistically significant impact in increasing the use of digitized information in radiology resident education.     

2001 Joseph E. and Nancy O. Whitley Award. Teaching noninterpretive skills to radiology residents: a collaborative effort between the American College of Radiology and the Association of Program Directors in Radiology.

RATIONALE AND OBJECTIVES: The authors' purpose was to develop, implement, and evaluate a series of videotapes for teaching noninterpretive skills to radiology residents. MATERIALS AND METHODS: An Association of Program Directors in Radiology/American College of Radiology working group developed a series of seven videotapes to teach residents job search and contracting skills; business aspects of radiology; American College of Radiology standards, accreditation programs, and appropriateness criteria; critical thinking skills; ethics; service orientation and interpersonal skills; and medical organizational politics. Residents viewed the videotapes, evaluated the learning experience, and completed both pre- and posttests. RESULTS: From 85 to 264 residents completed each of six pre- and posttests. Mean total pretest scores ranged from 55.0% to 76.9% and improved significantly (P < .05) to 73.8% to 94.9% on posttests. Each videotape was evaluated by 90-336 residents using a Likert-type scale, with a rating of 5 representing the most positive response. The proportion of residents who gave each tape an overall rating of either 4 or 5 varied from 75.9% to 95.9%. CONCLUSION: Significant resident learning occurred as a result of viewing the videotapes. Program director support of the learning experience was strong, and use of the videotapes as a teaching modality received strong support by residents. Local moderators and question-and-answer sessions would optimize the learning experience.     

Correlation between radiology resident rotation performance and examination scores

RATIONALE AND OBJECTIVES: The authors' purpose was to determine whether there is a relationship between subjective assessment of radiology resident performance on individual rotations and objective assessment of radiology resident performance on the American College of Radiology (ACR) in-training and American Board of Radiology (ABR) written examinations. MATERIALS AND METHODS: Records of 81 radiology residents completing their residency between 1991 and 2000 were reviewed. Mean scores from all rotation evaluation forms obtained during the study period were calculated for each residency year. The means of the overall raw scores and percentiles obtained on the annual ACR in-training examinations during the first 3 years of residency and of the written portion of the ABR examination taken during the 4th year of residency were also determined. Rotation evaluation scores were then compared to examination scores obtained during the same year of residency, and correlation coefficients were obtained. RESULTS: In the 2nd, 3rd, and 4th years of radiology residency, there is positive correlation between rotation evaluation scores and overall scores from the corresponding ACR in-training examination and written portion of the ABR examination taken during the same year. In contrast, in the 1st year of residency, resident rotation evaluation scores do not correlate with ACR in-training examination scores. CONCLUSION: Residents who are perceived as doing well on their rotations after the 1st year of residency are more likely to do well on standardized written examinations.     

Results of the 2001 survey of the American Association of Academic Chief Residents in Radiology

RATIONALE AND OBJECTIVES: The American Association of Academic Chief Residents in Radiology (A3CR2) conducts an annual survey of residency training programs. The survey data allow comparative analyses to be performed among training programs regarding resident education, benefits, clinical demands, and other resident-related issues. MATERIALS AND METHODS: Questionnaires were mailed to all accredited programs registered in the A3CR2 database (150 total programs). The yearly questions address demographic information concerning the individual programs including academic affiliation, number of radiologic examinations performed per resident, changes in the number of residents per program, and financial compensation. The rotating questions, which are revisited every 4 years, focused on preparation for the American Board of Radiology examinations, educational issues, and employment outlook. RESULTS: Completed surveys were received from 55 programs (37.3%). Among the responses, three issues prevailed: (a) increased number of examinations performed per resident, with resultant decrease in educational time, (b) decreased quality of education in all sections of radiology and decreased number of educational conferences, and (c) improved employment outlook, but continued trend of fewer residents choosing a career in academics. CONCLUSION: The current boom in the radiology job market and the increased number of radiologic examinations performed annually appear to adversely affect radiologic education through a decreased number of conferences and an increased number of radiologic examinations performed per resident.     

Part I: preparing first-year radiology residents and assessing their readiness for on-call responsibilities

RATIONALE AND OBJECTIVES: The aim of the study is to evaluate the effectiveness of an Emergency Radiology (ER) Core Curriculum training module and a Digital Imaging and Communications in Medicine (DICOM)-based interactive examination system to prepare first-year (postgraduate year 2 [PGY-2]) radiology residents and assess their readiness for taking overnight radiology call. MATERIALS AND METHODS: Institutional review board approval was obtained, and the study was compliant with Health Insurance Portability and Accountability Act (HIPAA) regulations. A dedicated month-long ER curriculum was designed to prepare new radiology residents for overnight radiology call that includes interpretation of off-hour urgent and emergent studies without immediate direct attending supervision. Lectures of the curriculum, provided by department staff, were based on the American Society of Emergency Radiology core curriculum. The lecture series was implemented after PGY-2 residents had completed formal introductory resident rotations during their first 6 months of training. A DICOM-based interactive computer-based testing module was developed and administered at the end of the lecture series. The module consisted of 19 actual emergency department cases with entire series of images, simulating an on-call setting. Tests were scored by two staff members blinded to resident identifying information. Upper-level residents also were tested, and comparison was made between first-year and upper-level resident test scores to determine the effectiveness of the test in determining first-year resident preparedness for call. Statistical analysis of results was performed by using t-test (P < .05). RESULTS: All residents in the residency program present during the month (nine PGY-2, six PGY-3, seven PGY-4, seven PGY-5 residents) attended the lecture series and finished the testing module at the end of the lecture series. Of 19 actual emergency cases on the testing module, five cases were neuroradiology, three cases were thoracic imaging, eight cases were body imaging, and three cases were musculoskeletal. PGY-2 residents scored an average of 73.0% (range, 63.2%-81.6%) of total points possible. PGY-3 residents scored an average of 76.8% (range, 68.4%-86.8%); PGY-4 residents scored an average of 77.4% (range, 65.8%-100%), and PGY-5 residents scored an average of 81.2% (range, 68.4%-94.7%). There was no statistically significant difference in scores according to level of training. CONCLUSION: First-year radiology residents who underwent 6 months of formal radiology training followed by an intensive ER lecture series before taking overnight call had scores similar to upper-level colleagues on an interactive computer-based ER simulation module.     

Editorial for anniversary issue of European Radiology

This special anniversary issue of European Radiology illustrates the wide range of topics which this journal now covers and charts some of the key recent advances in radiology over the last two decades. It includes several review articles in which each author considers some recent groundbreaking papers published in this journal and relates them to developments within their area of interest.     

Global Radiology Training Prevalence Among Radiology Residency Programs

ObjectiveGlobal Radiology aims to enhance access to medical imaging services and education, worldwide. To date, few reports have evaluated Global Radiology Training (GRT) in radiology residency programs. Here, we examined how radiology residency programs perceive and incorporate GRT into their curriculum, and how this information is promoted online.MethodsTwo methods were used to examine the current state of GRT. First, radiology residency program directors (identified via the Association of Program Directors in Radiology) were surveyed on topics including: Electives, institutional partnerships, resident and faculty involvement, inquiry by prospective residents, and barriers to implementation. Second, radiology residency program websites (n = 193) were examined for existing GRT on the programs’ publicly available webpages.ResultsThere were 62 survey responses (response rate of 19%). Thirty-eight percent (24/62) of residency programs offered a Global Radiology elective to their residents within the past academic year and 27% (17/62) of programs have active affiliations with medical institutions outside of the United States. Eighty-four percent of program directors (52/62) received questions from residency applicants regarding opportunities to participate in Global Radiology. Furthermore, only 13% (26/193) of all radiology residency programs listed at least one GRT elective on their webpage.DiscussionGRT in radiology residency is more widely available than previously reported and has been underrepresented on residency program websites. In the present survey, the majority of radiology residency program directors reported that radiology is an important component of Global Health, one-third of whom have already incorporated the subject into their curriculum. However, most common barriers to GRT include, perceived lack of time in the curriculum and lack of faculty interest. The high prevalence of inquiry from residency program applicants about GRT suggests that it may be a notable factor for applicants during the ranking process. Programs build up GRT may choose to share related information seeking to may choose to emphasize work in Global Radiology on their program webpages.     

Proposed program guidelines for pregnant radiology residents: a project supported by the American Association for Women Radiologists and the Association of Program Directors in Radiology

RATIONALE AND OBJECTIVES: Written institutional policies governing radiation exposure and work responsibilities for pregnant radiology residents are not uniform and often are nonexistent. Standardized program guidelines would allow residents and program directors alike to prepare for a resident pregnancy with objectivity and consistency. MATERIALS AND METHODS: The American Association for Women Radiologists (AAWR) launched a task force to revisit guidelines for the protection of pregnant residents from radiation exposure during training. We conducted two surveys of the Association of Program Directors in Radiology (APDR) membership. Survey 1 was designed to learn about existing program and institutional policies and to assess the need for and interest in standardized guidelines that would address radiation exposure and work responsibilities for pregnant radiology residents. Based on those responses, we drafted a set of program guidelines incorporating policies contributed by responding program directors. Our follow-up APDR survey, survey 2, was conducted to determine opinions and acceptance of the drafted program guidelines. Each survey was analyzed by using a proportion of means test. RESULTS: Fifty-five of 156 program director APDR members (35%) responded to survey 1. Only half the respondents had formal written policies at their respective institutions. Review of submitted policies showed widely divergent opinions about appropriate policies for pregnant radiology residents. Most (34/52; 75%) supported the development of standardized guidelines. In survey 2, 38/73 responding APDR members (53%) offered their opinions and comments on our drafted guidelines. Approximately 90% agreement was catalogued on 13 of 18 items (72%); a majority (>60%) agreed on all points, even the most controversial points concerning fluoroscopy. CONCLUSION: A minority of radiology residency programs have written policies addressing pregnancy during training. With expressed support from a majority of responding program directors, we have developed and present here proposed program guidelines for pregnant radiology residents to serve as a framework for radiology residents and program directors alike.     

How well does applicant rank order predict subsequent performance during radiology residency?

RATIONALE AND OBJECTIVES: Residency selection committees expend substantial time and resources on assessing the quality of residency applicants to derive an appropriate rank order for the National Residency Matching Program. The authors determined whether there is a relationship between the rank number or rank percentile of applicants selected for a residency training program and subsequent radiology residency performance. MATERIALS AND METHODS: Records of radiology residents completing their residency between 1991 and 1998 were reviewed. Available rank numbers and rank percentiles for each resident were compared with subsequent performance, as assessed subjectively by 4th-year radiology rotation evaluation forms and retrospective recall of four senior faculty members and objectively by numerical and percentile scores on the written portion of the American Board of Radiology (ABR) examinations. Correlation coefficients were obtained for each comparison. RESULTS: Rank number and rank percentile were not significantly correlated with 4th-year resident rotation evaluations or ABR written examination scores or percentiles. A small correlation existed between rank order and retrospective evaluation of resident performance by the four senior faculty. CONCLUSION: Applicant rank number and rank percentile do not correlate with subsequent radiology residency performance as assessed on rotation evaluation forms or the ABR written examinations.     

Factors affecting attending agreement with resident early readings of computed tomography and magnetic resonance imaging of the head, neck, and spine

RATIONALE AND OBJECTIVES: This study examines the joint effect of several factors on radiology resident performance in the task of interpreting after-hours neuroradiology examinations. MATERIALS AND METHODS: As part of a quality assessment process, we conducted a prospective evaluation of all (N = 21,796) after-hours preliminary readings of neuroradiology examinations performed by radiology residents over a 62-month period at our academic medical center. Each reading was scored by the interpreting neuroradiologist as "agree," "disagree with minimal clinical impact," and "disagree with significant clinical impact." Coded resident and attending identities were also recorded for each case along with modality, body area studied, and the date of examination. These raw data were used to create an analytic data set with level of resident/attending agreement as the outcome and six predictors, including two date-derived variables: months 1-62 representing when the case occurred during the study and quartiles 1-4 accounting for the timing of the case in each resident's own experience. Cross tabulations, plots, bivariate statistics, and logistic regression were used to examine the relationships between study variables and the outcome (level of agreement). RESULTS: Over about 5 years of the study, the absolute number of significant disagreements remained stable at about three per month. The total caseload increased at a rate of 4.1 per month with most of the increase falling into the agree category, whereas the minimal disagreements actually decreased slightly (0.2 per month). In the logistic model for disagreement, three of the factors accounted for most of the variance: attending (61%), resident (15%), and month (15%). Study type (modality and area examined) accounted for another 10%. There was no significant contribution from the variable (quartile) constructed to test for individual resident learning during the on-call experience. CONCLUSION: Although residents differ somewhat in the extent of attending agreement with their on-call work, evaluation or remediation made on the basis of simple comparison of these rates should be done with caution. Improved agreement over time seems to be a collective experience shared by residents.     

Radiology resident interpretation of on-call CT pulmonary angiograms

RATIONALE AND OBJECTIVES: To evaluate the interpretation of computed tomographic pulmonary angiograms performed outside of regular reporting hours, comparing the initial interpretation by the radiology resident to the attending radiologist. MATERIALS AND METHODS: Records for 840 consecutive computed tomographic pulmonary angiograms (CTPA) performed outside of regular reporting hours at two tertiary referral centers from January 1, 2004-December 31, 2005 were reviewed. The preliminary interpretation by the on-call radiology resident was compared to the subsequent final report issued by a subspecialty trained chest radiologist. Studies were stratified as positive, negative, or equivocal for pulmonary embolus. Cases with discordant interpretations or negative CTPA were reviewed to determine impact on clinical outcome. Patients were followed up to 12 months after CTPA to document any subsequent thromboembolic event. RESULTS: Sixteen percent (131/840) of CTPAs were reported positive by the staff radiologist. There was agreement in 90% (752/840) of studies (P = .76, 95% confidence interval, 0.71-0.81) with 86% (114/133) agreement for studies interpreted as positive by residents, 95% (582/612) for studies interpreted as negative by residents, and 63% (60/95) for studies interpreted as equivocal by residents. Studies of optimal quality had higher interobserver agreement than studies of suboptimal quality (P < .0001). In-patient studies were more likely to be positive than emergency room patients (20% vs. 13%) (P = .004). No adverse clinical outcomes were attributed to discordant interpretations. CONCLUSIONS: Radiology residents provide a high level interpretation of on-call CTPA studies, achieving good concordance with the attending radiologists' assessment.