Still Coming Out of the Dark: Enduring Effects of Simulation-Based Communication Skills Training for Radiology Residents—Four-Year Follow-Up

PurposeTo evaluate the long-term efficacy of simulation-based communication skills training for radiology residents.Method and MaterialsThe simulation-based communication skills training curriculum was developed in 2014. The curriculum included a teaching module based on the essential elements of communication. Two sets of 6 communication scenarios encountered by radiologist were created. First and fourth year radiology residents reviewed the teaching module and completed the 6 simulated scenarios. They then underwent debriefing sessions, received faculty and staff evaluations. Four years later, the former first year residents (now fourth years) reviewed the teaching module again and repeated the simulation. They again underwent debriefing sessions after the simulation. This time the residents’ communication skills were evaluated by faculty and staff.ResultsA total of 5 residents participated in this simulation-based skills training. The resident performance 4 years after initial training show not only that residents maintained their improved scores, but also that their scores improved further as compared to after the initial training. The average overall score for all but 1 resident increased at the 4 year follow-up simulation. From 2014 to 2018, the average score of all the residents increased from 72.4% to 81.4%. Comparison of the average scores of each student across 6 stations from 2014 to 2018 showed a statistically significant difference between the scores after 4 years (P = 0.014).ConclusionsSimulation-based communication skills training is effective and long lasting.     

Film screening by physician assistants in diagnostic radiology.

Data are presented which indicate that specially selected ART students can be trained to evaluate radiographs as to the presence or absence of significant pathology with an accuracy equivalent to that of experienced staff radiologists. The methods employed in this training program-namely, stressing perceptual recognition of normal and abnormal findings-may be beneficial in the early months of training of diagnostic radiology residents to hasten achievement of an acceptable perceptual performance. Test films, used in this study as an evaluation technique, may also be useful to assess residents' progress.     

Measurement and prediction of diagnostic performance during radiology training

Faculty evaluations of residents' diagnostic performance in radiology subspecialty rotations were examined in two studies in order to ascertain whether the relative capabilities of residents change during training and to predict residents' diagnostic performance using a three-dimensional form perception test. In the first study, numeric ratings by faculty members were averaged to provide interpretive/diagnostic scores for each of 16 residents in each of 5 consecutive half-years. Sixty-seven percent of the relative differences among residents' diagnostic proficiencies persisted during training. The magnitude of these unchanging differences between individuals in diagnostic performance strongly favors resident selection based upon diagnostic potential. An additional 22% of the relative differences correlated with an abrupt change in the rank order of residents in diagnostic performance at the beginning of the second year of residency. This rearrangement of ranks may have resulted from an abrupt change in the tasks and expectations assigned to residents. In the second study, correlations between scores on a recently described Form Test and monthly faculty ratings of diagnostic performance were computed. An average diagnostic performance score for each resident was generated for all rotations combined, each of eleven subspecialties, and all rotations completed within a given half-year. Scores on the Form Test correlated well with these combined diagnostic scores, further substantiating results reported previously. Form Test scores were highly correlated with subspecialty diagnostic performance scores in subspecialties using cross-sectional imaging methods, such as neuroradiology. Although Form Test scores were poorly correlated with half-year diagnostic performance during the first year, they were highly correlated with performance beginning in the second year.     

Part I: preparing first-year radiology residents and assessing their readiness for on-call responsibilities

RATIONALE AND OBJECTIVES: The aim of the study is to evaluate the effectiveness of an Emergency Radiology (ER) Core Curriculum training module and a Digital Imaging and Communications in Medicine (DICOM)-based interactive examination system to prepare first-year (postgraduate year 2 [PGY-2]) radiology residents and assess their readiness for taking overnight radiology call. MATERIALS AND METHODS: Institutional review board approval was obtained, and the study was compliant with Health Insurance Portability and Accountability Act (HIPAA) regulations. A dedicated month-long ER curriculum was designed to prepare new radiology residents for overnight radiology call that includes interpretation of off-hour urgent and emergent studies without immediate direct attending supervision. Lectures of the curriculum, provided by department staff, were based on the American Society of Emergency Radiology core curriculum. The lecture series was implemented after PGY-2 residents had completed formal introductory resident rotations during their first 6 months of training. A DICOM-based interactive computer-based testing module was developed and administered at the end of the lecture series. The module consisted of 19 actual emergency department cases with entire series of images, simulating an on-call setting. Tests were scored by two staff members blinded to resident identifying information. Upper-level residents also were tested, and comparison was made between first-year and upper-level resident test scores to determine the effectiveness of the test in determining first-year resident preparedness for call. Statistical analysis of results was performed by using t-test (P < .05). RESULTS: All residents in the residency program present during the month (nine PGY-2, six PGY-3, seven PGY-4, seven PGY-5 residents) attended the lecture series and finished the testing module at the end of the lecture series. Of 19 actual emergency cases on the testing module, five cases were neuroradiology, three cases were thoracic imaging, eight cases were body imaging, and three cases were musculoskeletal. PGY-2 residents scored an average of 73.0% (range, 63.2%-81.6%) of total points possible. PGY-3 residents scored an average of 76.8% (range, 68.4%-86.8%); PGY-4 residents scored an average of 77.4% (range, 65.8%-100%), and PGY-5 residents scored an average of 81.2% (range, 68.4%-94.7%). There was no statistically significant difference in scores according to level of training. CONCLUSION: First-year radiology residents who underwent 6 months of formal radiology training followed by an intensive ER lecture series before taking overnight call had scores similar to upper-level colleagues on an interactive computer-based ER simulation module.     

Communication in the diagnostic mammography suite: implications for practice and training

RATIONALE AND OBJECTIVES: The diagnostic mammography suite is a microcosm of challenging physician-patient communication in radiology. Little has been written about communication practices in the diagnostic mammography suite, the effect of this communication on both physicians and patients, and implications for radiology training programs. We surveyed radiology residents and staff about communication training, practices, and experiences communicating directly with patients in the diagnostic mammography suite. MATERIALS AND METHODS: We asked the membership of the Association of Program Directors in Radiology to disseminate surveys to radiology residents and staff radiologists in their institutions. We analyzed response frequencies and correlations. RESULTS: We received responses from 142 residents and 120 staff radiologists. More than half of staff respondents spoke personally with every patient who had an abnormal diagnostic mammogram; 37% felt they had inadequate time to do so. Most residents and staff highly rated their own communication skills and confidence in ability to explain results and respond to patients' emotions, but experienced stress doing so. A majority of respondents reported no formal communication skills education after medical school. Twenty-nine percent of staff respondents regularly observed residents' communication with patients and 39% of residents reported receiving feedback about their communication. Residents' opportunities to observe staff communicate with a patient and to receive feedback on their own patient interactions were correlated with self-rated communication skill and confidence in ability to respond to patients' emotions (P < .05). CONCLUSIONS: Radiologists engage in challenging and stressful patient communication interactions. There is a paucity of educational curricula on interpersonal and communication skills in radiology. This has implications for both patient and physician satisfaction and patient outcomes.     

Supervision of residents by faculty radiologists using home workstations

Our purpose was to determine if a home-based faculty radiologist equipped with a high-resolution workstation could add new information to residents' readings on overnight computed chest images that was equivalent to the new information generated by faculty reviewers inside the hospital. Teleconferencing software was installed on home workstations for online supervision of residents by faculty on chest images from a cardiothoracic intensive care unit. Critical observations that could affect patient care were recorded by first-year radiology residents before and after teleconferencing with the home-based radiologist. The amount of information added was compared with that which was added on the same 50 images through direct consultations with faculty inside the medical center. The amount of critical information that was added by teleconferencing with a chest radiologist at home was equivalent statistically to the information added through direct supervision of residents by faculty inside the hospital. Teleconferencing resulted in 149 changes in critical image findings as reported initially by the residents, out of 800 possible findings on 50 chest images, as opposed to 142 changes in residents' readings by faculty inside the medical center. Faculty subspecialists can supervise radiology residents effectively from their homes after hours, using high-resolution workstations and special teleconferencing software. Supported in part by a grant from the Richard F. and Eleanor W. Dwyer Fund for Excellence     

A web-based test of residents' skills in diagnostic radiology.

PURPOSE: To develop an objective, Web-based tool for evaluating residents' knowledge of diagnostic radiology. METHODS: We developed and tested a Web-based evaluation tool (the Diagnostic Radiology Skills Test) that consists of 3 tests, one in each of 3 domains of diagnostic radiology: chest, gastrointestinal, and musculoskeletal imaging. Each test comprises 30 cases representing a range of difficulty in the domain, including normal states, normal variants, typical cases of common diagnoses, and cases with more subtle findings. Cases are presented with a long menu of domain-specific possible diagnoses (response options), each coded for diagnostic appropriateness. Our subjects were 21 residents in postgraduate year (PGY) 2 to 5 and 11 experts in diagnostic radiology. Subjects accessed the tool via a Web site on our Web server. Residents' test results were compared for reliability and validity across domain, case, and training level. In addition, results were correlated with commonly used established and objective evaluation tools. RESULTS: The tool demonstrated consistent monotonic improvement in performance with training level. It showed acceptable reliability in discriminating between residents at different performance levels, both within and across training levels (r= 0.53 within level and 0.69 across levels). Test results also had concurrent validity against the American College of Radiology In-Training Examination, a widely accepted objective assessment tool (r = 0.65, P < 0.01), and 2 Objective Structured Clinical Examinations (OSCEs) focusing on diagnostic skills (r= 0.78 and r= 0.69, P < 0.01, respectively). CONCLUSIONS: Our study demonstrates the feasibility of a Web-based, standardized, objective assessment method for evaluating residents' performance.     

Using a sonographic simulator to assess residents before overnight call.

OBJECTIVE: The purpose of our study was to evaluate the effectiveness of a sonographic simulator in evaluating residents before their taking overnight call. MATERIALS AND METHODS: A sonographic simulator was used to teach and evaluate two consecutive classes of first-year radiology residents. A test consisting of 10 cases was given to each resident. Tests were scored with respect to image quality, measurement accuracy, and interpretation of results. Constructive feedback for improvement and additional training before taking call were provided, as needed. Surveys were given before and after the study to evaluate perceived knowledge and skills. RESULTS: In the first year of the study, five of eight residents scanned appropriately, giving reasonable differential diagnoses, whereas three residents performed suboptimally and were given feedback, additional training, or both. The sonography curriculum was restructured based on initial resident performance. In the subsequent year, all eight residents performed satisfactorily on the test. Comparing the 2 years of the study, mean test scores increased from 3.5 to 4.0 (p > 0.05) for abdominal test questions and from 3.4 to 4.2 (p < 0.05) for early obstetric and gynecologic test questions. The residents' self-assessment of knowledge and scanning ability also significantly improved. CONCLUSION: Sonographic simulation allows objective assessment and identification of weaknesses. These weaknesses can then be addressed before taking call, with resultant improved resident education and the presumed benefit of improved patient care.     

Home teleconsultations with faculty subspecialists on chest images from an intensive care unit

Study objective: To determine whether teleconsultations by first-year radiology residents with faculty thoracic radiologists, using diagnostic-quality teleradiology workstations in the radiologists' homes, would add clinical value for the acute management of intensive care unit patients after regular working hours. Design and setting: First-year radiology residents recorded key findings on new computed chest radiographs from 173 cardiothoracic intensive care unit patients. After consulting with home-based thoracic radiologists on the same images via teleradiology, they recorded any revisions to their original interpretations. An interdisciplinary evaluation panel determined whether the revisions of the residents' initial readings after the teleradiology consultations would have influenced the acute clinical care of these patients. Measurements and results: In 119 of the 173 cases (69 %), differences in key findings on the chest images were observed between the first-year residents' preliminary readings and their revised readings after teleconsultation with a thoracic radiologist. The evaluation panel determined that the changes in key findings after the teleconsultations could have influenced acute patient care in 86 of the 173 cases (50 %). Conclusions: Through rapid teleradiology consultations with residents, focusing on the key findings on newly-obtained computed chest images, home-based thoracic radiologists provided information of added clinical value for the acute management of cardiothoracic intensive care unit patients in one-half of the cases studied. Diagnostic workstations in the homes of faculty subspecialists may enable first-year radiology residents on night or weekend duty to obtain clinical supervision from faculty subspecialists that approximates more closely the level of supervision that they receive during regular working hours.     

Effect of a computer-aided diagnosis system on clinicians' performance in detection of small acute intracranial hemorrhage on computed tomography

RATIONALE AND OBJECTIVES: To analyze the effect of a computer-aided diagnosis (CAD) system on clinicians' performance in detection of small acute intracranial hemorrhage (AIH) on computed tomography (CT). MATERIALS AND METHODS: The authors have developed a CAD scheme that used both image processing techniques and anatomic knowledge based classification system to improve diagnosis of small AIH on CT. A multiple-reader, multiple-case receiver operating characteristic (ROC) study was performed. Twenty clinicians, including seven emergency physicians, seven radiology residents, and six radiology specialists were recruited as readers of 60 sets of brain CT, including 30 cases that show AIH smaller than 1 cm, and 30 controls. Each reader read the same 60 cases twice, first without, then with the prompts produced by the CAD system. The clinicians ranked their confidence in diagnosing a case of showing AIH, which produced the ROC curves. RESULTS: Significantly improved performance is observed in emergency physicians, average area under the ROC curve (Az) increased from 0.8422 to 0.9294 (P = .0107) when they make the diagnosis without and with the support of CAD. Az for radiology residents increased from 0.9371 to 0.9762 (P = .0088). Az for radiology specialists increased from 0.9742 to 0.9868, but was statistically insignificant (P = .1755). CONCLUSIONS: CAD can improve the clinicians' performance in detecting AIH on CT. In particular, emergency physicians can benefit most from the CAD and improve their performance to a level approaching that of the average radiology residents.     

Resident vs. attending interpretation of on-call studies: clinical impact

Purpose: This study was performed to determine whether significant changes to patient treatment plan or outcome result from discrepancies between on-call radiology residents and follow-up attending radiologists in their interpretation of examinations. Methods: For 70 days we recorded on-call radiology residents' readings of all computed tomography and ultrasound examinations performed in our institution and the follow-up attending radiologists' readings of these same examinations. A chart review was performed to determine whether interpretation discrepancies changed the treatment plan and clinical outcome. Results: Eight-hundred thirty-four examinations met the study guidelines. The overall discrepancy rate was 5.16 %. Of these discrepancies, 6.98 % affected the treatment plan (0.36 % of all 834 studies) and none affected the clinical outcome. Conclusion: Where there is a discrepancy between interpretation of computed tomography and ultrasound after hours by on-call radiology residents and follow-up readings by attending radiologists, this discrepancy has no significant effect on the immediate or long-term care of patients.     

Resident interpretation of emergency computed tomographic scans

Our busy, urban emergency room is staffed by radiology residents after working hours. To determine the accuracy of our residents' interpretations of emergency cranial computed tomographic (CT) scans, the authors reviewed the preliminary reports of our residents for a two-month period. A total of 289 cranial CT scans were retrospectively reviewed and the resident interpretation judged acceptable, minor error, moderate error, or major error. Six of 289 neurologic examinations (2%) had moderate (4) or major (2) errors. The mistakes all involved misinterpretation of cerebral hemorrhage. The 98% accuracy in interpretation of cranial CT is higher than the accuracy reported with emergency plain film (PF) interpretation.     

Impact of pre-radiology clinical years on resident performance

In 1971, the American Board of Radiology dropped the requirement for at least one pre-radiology clinical year (PRCY) of training. However, requiring a clinical year for radiology residents remains a controversial issue. We reviewed 896 rotation evaluation forms involving 62 residents to see if a difference in performance could be detected between residents who entered training without or with at least one PRCY. Forty-five residents had no previous clinical training, whereas 17 had one or more PRCYs. No significant differences were detected in resident performance between non-PRCY and PRCY residents, nor were differences found by sex, age, presence of non-medical postgraduate degrees, or scores on the first part of the National Board of Medical Examiners. No study of PRCY versus non-PRCY residents can be adequately controlled; thus the failure to show a difference in performance must be interpreted with caution.     

Emergency radiology services at medical schools in the United States.

RATIONALE AND OBJECTIVES. Little has been published on the delivery of emergency radiologic services in academic radiology training programs. METHODS. The author surveyed 127 medical schools in the United States concerning aspects of radiology services for their emergency rooms, including who interprets images, what training in emergency radiology is provided, and problems with film retrieval. RESULTS AND CONCLUSIONS. Emergency department radiographs most often are initially interpreted by a radiology resident and subsequently reviewed by a faculty radiologist who does not have a major interest in or time commitment to emergency radiology. Most schools describe problems such as disappearance of emergency department films and a paucity of provided clinical information. Only one third of schools provide formal instruction in emergency radiology for their radiology residents.     

Impact of PACS on the education of radiology residents: the residents' perspective

RATIONALE AND OBJECTIVES: Because digital imaging and the picture archiving and communication system (PACS) are replacing radiographic film, the effect of PACS on residents' perceptions and their educational experience was investigated. MATERIALS AND METHODS: Residents taking part in large diagnostic radiology training programs at two hospitals were surveyed. Approximately 75% of radiographic studies were reviewed with the use of PACS at both hospitals. Survey topics included technical and didactic issues based on direct and indirect comparison with analog (conventional film) images. RESULTS: Fifty residents were polled (20 respondents). The majority has been using PACS for more than 1 year (14 of 20, 70%) to interpret 75%-100% of cases (11 of 20, 55%). The majority believed that PACS improved patient care (15 of 20, 75%) and their educational experience (15 of 20, 75%). A minority believed that increased patient throughput was harmful to the educational experience (five of 20, 25%) because it permitted attending radiologists to review cases too quickly (four of 20, 20%). Residents favored PACS over hard-copy images for ease of manipulation, resolution, and ability to see pathologic conditions and normal anatomic characteristics. CONCLUSION: Residents believe that PACS has positively affected their learning experience and does not negatively affect the quality of resident education.     

Radiology resident evaluation of head CT scan orders in the emergency department

BACKGROUND AND PURPOSE: Prior studies have revealed little difference in residents' abilities to interpret cranial CT scans. The purpose of this study was to assess the performance of radiology residents at different levels of training in the interpretation of emergency head CT images. METHODS: Radiology residents prospectively interpreted 1324 consecutive head CT scans ordered in the emergency department at the University of Arizona Health Science Center. The residents completed a preliminary interpretation form that included their interpretation and confidence in that interpretation. One of five neuroradiologists with a Certificate of Added Qualification subsequently interpreted the images and classified their assessment of the residents' interpretations as follows: "agree," "disagree-insignificant," or "disagree-significant." The data were analyzed by using analysis-of-variance or chi-squared methods. RESULTS: Overall, the agreement rate was 91%; the insignificant disagreement rate, 7%; and the significant disagreement rate, 2%. The level of training had a significant (P =.032) effect on the rate of agreement; upper-level residents had higher rates of agreement than those of more junior residents. There were 62 false-negative findings. The most commonly missed findings were fractures (n = 18) and chronic ischemic foci (n = 12). The most common false-positive interpretations involved 10 suspected intracranial hemorrhages and suspected fractures. CONCLUSION: The level of resident training has a significant effect on the rate of disagreement between the preliminary interpretations of emergency cranial CT scans by residents and the final interpretations by neuroradiologists. Efforts to reduce residents' errors should focus on the identification of fractures and signs of chronic ischemic change.     

Use of personal digital assistants in diagnostic radiology resident education

RATIONALE AND OBJECTIVES: Personal digital assistants (PDAs) are gaining widespread use in the medical community. We introduced a PDA-based mobile system that provides departmental and educational information with a seamless connection to the intranet. The objective of this study is to determine the impact a PDA has on educational resources (learning or data reference) brought to work and used at home by a radiology resident based on user surveys. MATERIALS AND METHODS: Survey was performed on 32 radiology residents in our department before and 6 months after the release of the PDA-based system. We assessed the changes in (1) sources of learning at home and at work, and in (2) data reference. The second survey also evaluated the usefulness of each component of the system. RESULTS: After the release of the PDA-based mobile system, the use of "digital books and references" as data references and educational resources that were brought to work every day significantly increased (P = .016, P < .0001, respectively). "Traditional books and references" remained the "most useful source in learning radiology"; however, "digital books and references" increased as the residents' first choice from 0% to 16% within 6 months of introducing the package (P = .125). CONCLUSION: The introduction of a PDA-based system consisting of educational and departmental information had a statistically significant impact in increasing the use of digitized information in radiology resident education.     

From Being a Radiologist to Watching a Radiologist: Impact of Filmless Operation on the Training of Radiology Residents

RATIONALE AND OBJECTIVES: The authors performed this study to investigate the impact of changing from a film-based image interpretation system to one using digital image workstations on the training of radiology residents in the interpretation of radiographs. MATERIALS AND METHODS: Data were collected during a period when a conventional system of image interpretation with hard-copy images and multiviewers was used and during a period when digital image workstations were used. During each period, it was noted whether the first interpretation of the radiographs was performed by a radiology resident, by an attending radiologist, or as a group effort including both an attending radiologist and a radiology resident(s). In addition, it was noted whether a radiology resident or an attending radiologist dictated the report. RESULTS: The proportion of images first interpreted by the radiology resident alone decreased from 38% (53 of 139) when using the conventional system to 17% (34 of 199) after the switch to interpreting images on the workstations (P = .001). During the film-based period, radiology residents dictated 45% of reports (141 of 312), but during the workstation period, radiology residents dictated only 4% of reports (24 of 667; P = .001). CONCLUSION: The authors observed a decrease in autonomous participation by radiology residents in image interpretation and dictation of reports and an increase in "group reading" after the switch from a film-based system to a workstation system.     

Clinical consequences of misinterpretations of neuroradiologic CT scans by on-call radiology residents

BACKGROUND AND PURPOSE: Studies have looked at the accuracy of radiologic interpretations by radiology residents as compared with staff radiologists with regard to emergency room plain films, emergency room body CT scans, and trauma head CT scans; however, to our knowledge, no study has evaluated on-call resident interpretations of all types of neuroradiologic CT scans. Both as a part of our departmental quality control program and to address concerns of clinical services about misinterpretation of neuroradiologic CT scans by on-call radiology residents, we evaluated the frequency of incorrect preliminary interpretations of neuroradiologic CT scans by on-call radiology residents and the effect of such misinterpretations on clinical management and patient outcome. METHODS: As determined by the staff neuroradiologist the next day, all potentially clinically significant changes to preliminary reports of emergency neuroradiologic CT scans rendered by on-call radiology residents were recorded over a 9-month period. A panel of neuroradiologists reviewed and graded all the changed cases by consensus. An emergency department staff physician reviewed medical records of all submitted cases to determine clinical consequences of the misinterpretations. RESULTS: Significant misinterpretations were made in 21 (0.9%) of 2388 cases during the study period. There was a significant change in patient management in 12 of the cases, with a potentially serious change in patient outcome in two cases (0.08%). CONCLUSION: On-call radiology residents have a low rate of significant misinterpretations of neuroradiologic CT scans, and the potential to affect patient outcome is rare.     

CT pulmonary angiography for the detection of pulmonary embolism: interobserver agreement between on-call radiology residents and specialists (CTPA interobserver agreement)

PURPOSE: The aim of this study was to prospectively determine interobserver agreement between on-call radiology residents and specialists in the interpretation of computed tomographic pulmonary angiography (CTPA). METHODS: CTPA examinations obtained between January 2002 and March 2003 were interpreted by a radiology resident on call and by two radiology specialists. Agreement was assessed using percentage of agreement between interpreters and by the kappa coefficient. Sensitivity of residents' interpretations was calculated by relating them to the interpretation of Specialist 1, which served as the gold standard. RESULTS: Of the 81 CTPA examinations evaluated, there was agreement of 93% and 91% for the diagnosis of pulmonary embolism (PE) and of 97% and 85% for the exclusion of PE with Specialist 1 and 2, respectively. The concordance between residents' interpretations and those of Specialist 1 was very high (kappa=.8), and with those of Specialist 2 was high (kappa=.7). In all cases of agreement between the two specialists, there was complete agreement between the specialists' and the residents' intepretations. CONCLUSIONS: Our study showed good to very good agreement of residents' interpretations with each of the radiology specialists. Therefore, relying on the residents' preliminary interpretations during after-hour calls is reasonable.