Radiologists' productivity in the interpretation of CT scans: a comparison of PACS with conventional film

OBJECTIVE: We compared radiologists' times in the interpretation of CT using hardcopy films with the interpretation using a soft-copy picture archiving and communication system (PACS) computer workstation. MATERIALS AND METHODS: One hundred CT examinations were selected at random and reviewed by four board-certified radiologists experienced in soft-copy interpretation. We performed time-motion analysis to determine the total time required to display, interpret, and dictate the individual findings of CT using conventional hard-copy interpretation on a viewbox and soft-copy interpretation, using a four-monitor high-resolution (2048 x 1536 pixel) workstation. RESULTS: Time-motion analysis showed a reduction of 16.2% in the overall time required for soft-copy interpretation of CT compared with that of film. Time savings with soft-copy interpretation were observed for all four participating radiologists. The benefit of soft-copy interpretation was increased for examinations in which there were comparison studies. CONCLUSION: We found that soft-copy interpretation of CT using a PACS workstation requires less time than interpretation using conventional film hung on a viewbox. The transition to filmless imaging has the potential to improve radiologists' productivity and report-turnaround time.     

Receiver-operating-characteristic study of chest radiographs in children: digital hard-copy film vs 2K x 2K soft-copy images.

Two methods are commonly used to visualize digital radiologic imaging data: (1) hard-copy viewing, in which the digital data are used to modulate the intensity of a laser beam that exposes an analog film and (2) soft-copy viewing, in which the digital data are converted to an analog video signal and presented on a CRT monitor. The film method allows new digital imaging systems to be easily integrated into conventional radiologic management and viewing methods. The second method, soft-copy viewing, allows digital imaging data to be managed and viewed electronically in a picture archiving and communication system (PACS). These PACS systems are hypothesized to have improved operational efficiency and enhanced image-analysis capabilities. The quality of soft-copy images is still not widely accepted. This article reports on the results of a large-scale receiver-operating-characteristic study comparing observers' performance in detecting various pediatric chest abnormalities on soft-copy 2048 x 2048K byte displays with their performance with digital laser-printed film from computed radiography. The disease categories studied were pneumothorax, linear atelectasis, air bronchogram, and interstitial disease. The selected data set included 239 images; 77 contained no proved abnormality and 162 contained one or more of the abnormalities mentioned. Seven pediatric radiologists participated in the study, two as judges and five as observers. Our results show no significant difference between viewing images on digital hard copy and soft copy for the detection of pneumothoraces and air bronchograms. A slight performance edge for soft copy was seen for interstitial disease and linear atelectasis. This result indicates that computed chest radiographs in children viewed in a soft-copy PACS environment should result in diagnoses similar to or slightly more accurate than those obtained in a laser-printed film-based environment.     

Ocular health among radiologists in the age of PACS: is it time for our profession to open its eyes to this issue in light of existing European legislation?

The regular use of visual display units (VDUs) at work has been shown to cause the development of a constellation of symptoms ranging from dry eyes to temporary myopia. European workers who use VDUs are now protected under detailed legislation enacted by the European Union (Directive 90/270/EEC). The use of picture archiving and communications systems, which are almost ubiquitous in European countries, means that, as a profession, radiologists fall under the remit of this legislation. This paper aims to assess the impact that full implementation of this law would have on a radiologist''s practice and to more broadly examine the issue of eye care as an occupational health issue in radiology. The authors conclude that eye care in the setting of regular VDU use among radiologists is an important quality control and occupational health issue. There is a clear legal basis requiring employers to provide regular eye examinations and reporting breaks. In the absence of leadership from employers on this issue individual radiologists have a responsibility to ensure that their work practices reflect the legal situation and minimise the effect of eye strain on their performance.The regular use of visual display units (VDUs) at work has been shown to cause the development of a constellation of symptoms ranging from dry eyes to temporary myopia. European workers who use VDUs are now protected under detailed legislation enacted by the European Union (Directive 90/270/EEC) [1]. The use of picture archiving and communications systems (PACS), which are almost ubiquitous in European countries, means that, as a profession, radiologists fall under the remit of this legislation. This paper aims to assess the impact that full implementation of this law would have on a radiologist''s practice and to more broadly examine the issue of eye care as an occupational health issue in radiology.     

PACS workstation design

This paper covers some of the recent concepts in designing a digital imaging workstation in a multimodality Picture Archiving and Communications Systems (PACS) network. A workstation in a multimodality PACS network must access, display, and analyze digital images from different imaging modalities with very different formats. The user interface should allow clinicians with minimal or no computer manipulation skills to use complex analysis tools. General guidelines of a graphics oriented user interface, based on windows and icons, are proposed. Instantaneous (real-time) response in the primary display and processing functions is vital for user acceptance. The hardware architectural concepts to achieve such a performance speed are described. Finally, a workstation environment conducive to comfortable viewing by the radiologists is discussed.     

Interobserver agreement and performance score comparison in quality control using a breast phantom: screen-film mammography vs computed radiography

Our objective was to evaluate interobserver agreement and to compare the performance score in quality control of screen-film mammography and computed radiography (CR) using a breast phantom. Eleven radiologists interpreted a breast phantom image (CIRS model X) by four viewing methods: (a) original screen-film; (b) soft-copy reading of the digitized film image; (c) hard-copy reading of CR using an imaging plate; and (d) soft-copy reading of CR. For the soft-copy reading, a 17-in. CRT monitor (1024x1536x8 bits) was used. The phantom image was evaluated using a scoring system outlined in the instruction manual, and observers judged each object using a three-point rating scale: (a) clearly seen; (b) barely seen; and (c) not seen. For statistical analysis, the kappa statistic was employed. For "mass" depiction, interobserver agreement using CR was significantly lower than when using screen-film ( p<0.05). There was no significant difference in the kappa value for detecting "microcalcification"; however, the performance score of "microcalcification" on CR hard-copy was significantly lower than on the other three viewing methods ( p<0.05). Viewing methods (film or CR, soft-copy or hard-copy) could affect how the phantom image is judged. Paying special attention to viewing conditions is recommended for quality control of CR mammograms.     

Reliability of soft-copy versus hard-copy interpretation of emergency department radiographs: a prototype study

OBJECTIVE: The purpose of this study was to compare the diagnostic reliability of hard-copy and soft-copy interpretation of radiographs obtained in the emergency department using a methodology for evaluating imaging systems when independent proof of the diagnosis is not available. MATERIALS AND METHODS: We collected radiographs from a stratified sample of 100 patients seen in the emergency department. The images were obtained using computed radiography, and the digital images were printed on film and stored for display on a workstation. A group of seven experienced radiologists reported the cases using both film and the workstation display. The results were analyzed using mixture distribution analysis (MDA). RESULTS: The reliability expressed as the percentage of agreement of a typical observer relative to the majority was computed from the MDA. The result was 90% for both hard copy and soft copy with bootstrap confidence intervals of 86-94%. CONCLUSION: We conclude that, in the emergency department, soft-copy interpretation is as reliable as hard-copy interpretation. The strength of this conclusion depends on the validity of the MDA approach as well as the extent to which the observer sample and case sample are representative of the emergency department.     

Efficacy of digital radiography for the detection of pneumothorax: comparison with conventional chest radiography.

As part of our continuing evaluation of the clinical applicability of digital radiography, we compared the abilities of radiologists to detect pneumothoraces on conventional chest radiographs with their performances when using three formats of digitally obtained images. Twenty-three frontal-view chest radiographs with pneumothoraces and 22 other chest radiographs, either normal or showing miscellaneous abnormalities, were interpreted by five experienced radiologists in each of four formats: conventional film-screen chest radiographs, small-format (17.8 x 21.6 cm) computed radiographs, large-format (35.6 x 43.1 cm) computed radiographs, and digital images viewed on an interactive electronic workstation. The receiver-operating-characteristic curve areas for each observer for the four types of images were compared by a z test on a critical ratio, and the mean sensitivity and specificity values were compared by the sign rank test. The mean areas under the receiver-operating-characteristic curves ranged from 0.869 for the digital workstation to 0.915 for film-screen images. The differences observed among formats were not statistically significant. Mean specificities also were not significantly different, ranging from 0.90 for large-format computed radiographs to 0.96 for the digital workstation. Mean sensitivity ranged from 0.65 for the digital workstation to 0.82 for film-screen images. Radiologists interpreting digital workstation images were significantly less sensitive in detecting pneumothoraces than with film-screen and small-format computed images (p = .06). In this study, radiologists detected pneumothoraces equally well on conventional film-screen radiographs and digital images printed on film; however, they detected pneumothoraces less well on electronic viewing consoles. This latter finding reflects an important practical difference in the working behavior of radiologists interacting with a digital workstation.     

Impact of PACS on the education of radiology residents: the residents' perspective

RATIONALE AND OBJECTIVES: Because digital imaging and the picture archiving and communication system (PACS) are replacing radiographic film, the effect of PACS on residents' perceptions and their educational experience was investigated. MATERIALS AND METHODS: Residents taking part in large diagnostic radiology training programs at two hospitals were surveyed. Approximately 75% of radiographic studies were reviewed with the use of PACS at both hospitals. Survey topics included technical and didactic issues based on direct and indirect comparison with analog (conventional film) images. RESULTS: Fifty residents were polled (20 respondents). The majority has been using PACS for more than 1 year (14 of 20, 70%) to interpret 75%-100% of cases (11 of 20, 55%). The majority believed that PACS improved patient care (15 of 20, 75%) and their educational experience (15 of 20, 75%). A minority believed that increased patient throughput was harmful to the educational experience (five of 20, 25%) because it permitted attending radiologists to review cases too quickly (four of 20, 20%). Residents favored PACS over hard-copy images for ease of manipulation, resolution, and ability to see pathologic conditions and normal anatomic characteristics. CONCLUSION: Residents believe that PACS has positively affected their learning experience and does not negatively affect the quality of resident education.     

Interobserver and intraobserver variability in the assessment of pulmonary nodule size on CT using film and computer display methods

RATIONALE AND OBJECTIVES: A critical element in determining biologic behavior of pulmonary nodules is volume and temporal volume change. We evaluate variability in nodule volume among readers and measuring methods. MATERIALS AND METHODS: 55 small (<2 cm) lung nodules were measured in long- and short-axis dimensions independently by 4 radiologists, using 3 methods: 1) hard copy, 2) GE Advantage Windows workstation (GE Healthcare, Milwaukee, WI), 3) Siemens IMACS workstation (Siemens Medical Systems, Iselan, NJ). Nodule margin was recorded as smooth, lobulated, or spiculated. Volume was calculated from diameter measurements. Variability in nodule volume was evaluated within each reader, between readers, and across measurement tools. RESULTS: Mean nodule short-axis diameter was 5.3 mm; mean long-axis diameter 7.2 mm. There was statistically significant variation among readers and measurement method for nodule volume. Volume was significantly larger using hard-copy measurements (51.9%-54.1% variation; P < .0001) than either workstation, and not different between workstations. There was greater intraobserver variability in volume using the hard-copy method, and no difference between workstation methods. Volumes based on measurements from one reader were consistently lower than those from other readers (P = < .001, .003, and .02); volume was consistently larger for another reader (P < .0001, .03, and .12). Reader agreement for nodule margin was good to excellent. CONCLUSION: Considerable interobserver and intraobserver variability in measuring nodules exists using hard-copy and computer tools. Since a small change in diameter indicates a much larger change in volume, this may be significant when using early repeat CT to follow small pulmonary nodules. Computer-aided diagnostic tools that reproducibly measure nodule volume are strongly needed.     

Enhanced visualization processing: effect on workflow.

RATIONALE AND OBJECTIVES: Soft-copy viewing of digital radiographs allows for image processing to improve visualization of anatomy and lesions, but it can take more time than film-based viewing. Enhanced visualization processing (EVP) was developed to increase the latitude of an image without reducing the vital contrast, potentially reducing the need for the radiologist to manipulate images. This study examined the influence of processing radiographic images with EVP on workflow in a picture archiving and communications system (PACS). MATERIALS AND METHODS: Portable computed radiographic chest images were obtained and processed either with EVP or without. A security camera with a videocassette recorder was positioned above the PACS workstation. Four radiologists reviewed the images during their normal work schedule. The current diagnostic image was used to determine if the case contained EVP or non-EVP images. The videotapes of the sessions were reviewed to determine diagnostic viewing times and how zoom and/or window and level manipulation was used. RESULTS: Viewing time was significantly longer for the non-EVP than the EVP cases. The difference occurred with all readers. Window and level manipulation was used on 35% of the EVP and 41% of the non-EVP images. Zoom was used on 64% of the EVP and 69% of the non-EVP images. Average time spent using zoom and window and level manipulation was significantly shorter with the EVP than with the non-EVP images. CONCLUSION: EVP of chest images displayed on PACS monitors significantly improved workflow as measured by viewing time. EVP decreased use of window and level manipulation and zooming and the amount of time each one was used.     

医学影像存档与通讯系统的开发与初步应用

目的 通过组建简便医学影像存档与通讯系统（ｐｉｃｔｕｒｅ ａｒｃｈｉｖｉｎｇ ａｎｄ ｃｏｍｍｕｎｉｃａｔｉｏｎ ｓｙｓｔｅｍｓ,ＰＡＣＳ）实现影像诊断设备的网络化,诊断报告书写计算机化,标准化。方法 ＣＴ,ＭＲＩ和Ｓｕｎ Ａｄｖａｎｔａｇｅ Ｗｉｎｄｏｗｓ１２．０工作站连接成医学数字影像传输（ＤＩＣＯＭ）网络;ＤＩＣＯＭ 与各图像浏览及诊断报告书写终连接成以太网网络;二者再通过集线器连接成ＰＡＣ     

Modular project of the Picture Archiving and Communication System (PACS). Preliminary clinical experience. II

In the previous paper in this volume the PAC System installed in the Radiology Department of the University of Trieste has been described and its advantages and limitations have been analyzed, mainly from an operational point of view. This paper deals with the clinical evaluation of the system in ordinary operative conditions. A series of cases with specific characteristics was monitored in order to reveal different performances in both diagnostic process and conclusions using the PACS viewing console (DW) vs. conventional CRT film images on alternators. In a first test, 100 routine (not pre-selected) brain CT cases were independently analyzed by 4 radiologists, each of them giving 2 interpretations of the same case, one based on film and the other on PACS. The data were analyzed by conventional statistical methods, showing a substantial agreement of the results obtained with the 2 modalities. A second test concerned the evaluation of 100 lumbar intervertebral disks by CT, with the same procedure as above. Four radiologists were again asked to decide on film and PACS images about normality, protusion, or herniation of the disks. The results demonstrate the possibility of adequately reporting on the PACS monitor and stress the need of an adequate training period and the efficacy of the image processing capabilities of the system.     

Soft-copy versus hard-copy interpretation of voiding cystourethrography in neonates, infants, and children

OBJECTIVE: Our aim is to determine whether agreement occurs between soft-copy (at a review workstation) and hard-copy (on laser-printed film) interpretations of fluoroscopic voiding cystourethrograms in neonates, infants, and children. SUBJECTS AND METHODS: Voiding cystourethrography was performed on 74 children (range, 2 weeks to 11 years old; mean, 3 years 6 months) for the evaluation of vesicoureteral reflux. The right and left ureters were scored separately by two observers on a scale of 0-5 using the international grading standard. Differences were tested for statistical significance with a marginal homogeneity test, and the strength of agreement was assessed using the kappa statistic. RESULTS: Of the 148 ureters evaluated, 39 showed vesicoureteral reflux and 109 showed no vesicoureteral reflux on both soft copy and hard copy. For 128 of 148 evaluations, interpretations of soft copy and hard copy produced agreement as to the grade of vesicoureteral reflux. For 11 of the 20 ureters with divergent interpretations, hard copy was scored one grade lower than soft copy; for the remaining nine ureters, hard copy was scored one grade higher than soft copy. No score differed by more than one grade. We found no statistically significant difference between soft-copy and hard-copy scores (p = .65), and agreement using the kappa statistic was substantial (.68). CONCLUSION: Soft-copy interpretation of voiding cystourethrograms is similar to hard-copy interpretation for vesicoureteral reflux.     

Filmlose Magnetresonanztomographie Vor- und Nachteile im Vergleich zur Filmbefundung

At our hospital, an average MRI examination comprises 170 images. These examinations were evaluated exclusively on digital reporting systems with two high-resolution monitors for a period of 1 year. A retrospective analysis of this reporting method in comparison with films on a conventional viewing wall was performed with subjective and objective data. Radiologists and technicians were requested to evaluate the system on a subjective scale from 1 to 5 for seven topics (image analysis, inspection, time for reporting, handling, comparison with previous reports, quality of PC laser prints, training period). Moreover, personal interviews were conducted and protocols taken. Patient and image frequency, film costs, data transfer time and amount of system failures were evaluated as objective data. The most important results were: Two thirds of the film checking radiologists prefer the conventional viewing walls over the computer monitors. However, 70% of the residents prefer using the computer monitor for reporting. Seventy percent of the interviewed radiologists considered comparison with former examinations on film very difficult. Digitizing of former MRI examinations was not a convincing method; printing on a standard PC laser printer was considered to yield insufficient quality. The different acceptance between radiologists and residents seems to be related to different experiences. The reduction in film costs (48.6%) will improve further with complete PACS installation in the whole hospital. Data transfer rates are still poor; further improvement of network performance is necessary for convenient work. One whole MRI examination and report could be stored on CD-ROM for a cost of less than 2 euros. This could be a future means of cheap archiving and documentation suitable for viewing on any PC with DICOM III viewer. Images and reports could stay with the patients as in the past.     

Comparison of 2048-line digital display formats and conventional radiographs: an ROC study

Observer performance tests were conducted to compare the effects on diagnostic accuracy of digital hard copy and video display formats versus conventional radiographic film. Digital images were obtained by digitizing conventional chest radiographs to a 2048 x 2048 matrix with a laser film scanner. Three digital display formats were used: laser-printed digital film, a 2048-line video monitor without user interaction, and a 2048-line video monitor with user interaction. Thirty-one posteroanterior chest radiographs, determined by consensus of four thoracic radiologists to contain septal lines (n = 11), parenchymal nodules (n = 7), nodules and septal lines (n = 7), or neither abnormality (n = 6), were used for the study. Images were interpreted by four radiologists in four separate viewing sessions. Diagnostic accuracy was determined by receiver-operating characteristic analysis for each observer with each viewing technique. No statistical differences in diagnostic accuracy, determined by the area under the receiver-operating-characteristic curve, were found between the analog film, the digital film, and the two video digital display formats. This preliminary study suggests that 2048-line digital displays may be an acceptable alternative to the traditional lightbox viewing method for the perception of these two abnormalities commonly seen on chest radiographs.     

Radiologists' reading times using PACS and using films: one practice's experience

RATIONALE AND OBJECTIVES: To measure the change in radiologists' productivity in terms of interpretation time per examination when using picture archiving and communication system (PACS) workstations in a particular private practice, Valley Radiologists, Ltd, as part of a feasibility study and subsequent business plan to implement a digital enterprise. MATERIALS AND METHODS: Time to process a series of exams was measured for 18 radiologists during an uninterrupted period of a working day. Radiologists in the practice served in multiple locations. The data were analyzed in aggregate and by modality (plain film, ultrasound, computed tomography, and magnetic resonance imaging). Average time per exam, with and without PACS, was measured for each modality. Regression analysis was used to determine the independent effect of PACS on radiologist productivity. RESULTS: The mean time to process an exam was 1.4 minutes (SE = 0.04) for plain film, 1.96 minutes (SE = 0.14) for ultrasound, 5.08 minutes (SE = 0.44) for computed tomography, and 6.83 minutes (SE = 0.31) for magnetic resonance imaging. Regression results indicate that PACS had no effect on the time taken to read a series of exams. CONCLUSIONS: When considering a PACS purchase or implementation, decrease in radiologists' time to process an examination may not be realized. In this specific practice setting, we did not find evidence that PACS workstations alone, without any other changes in workflow design, improved radiologists' interpretation time.     

Evaluation of a digital workstation for interpreting neonatal examinations. A receiver operating characteristic study.

RATIONALE AND OBJECTIVES. The purpose of this study is to compare the diagnostic accuracy of interpreting clinical neonatal radiographs using a commercially available digital workstation versus conventional radiographic images. METHODS. The case sample consists of 100 chest or abdominal radiographs from the neonatal intensive care unit in which diagnosis was confirmed. Four radiologists served as observers. During two initial reading sessions, half of the studies were viewed on digital radiography monitors and the other half by plain film. Observers indicated whether each patient had normal or abnormal findings and their degree of confidence in this judgment. Six weeks later, observers viewed cases on the alternate presentation system. Two statistical analyses were performed: the data from each observer were treated as a separate experiment in the first analysis; the data from all observers were combined using a new method in the second analysis. RESULTS. No differences between areas under receiver operating characteristic (ROC) curves for viewing on the picture archiving and communication system (PACS) console and plain film were found for any observer (0.86 versus 0.86, 0.89 versus 0.86, 0.88 versus 0.85, 0.83 versus 0.82). CONCLUSIONS. The study suggests that for pediatric plain film images, video images offer diagnostic information comparable with that of conventional radiographs for neonatal examinations.     

Detection rates in pediatric diagnostic imaging: A picture archive and communication system compared with a web-based imaging system.

OBJECTIVE: This prospective study assesses whether there are differences in accuracy of interpretation of diagnostic images among users of a picture archive and communication system (PACS) diagnostic workstation, compared with a less costly Web-based imaging system on a personal computer (PC) with a high-resolution monitor. METHODS: One hundred consecutive pediatric chest or abdomen and skeletal X-rays were selected from hospital inpatient and outpatient studies over a 5-month interval. They were classified as normal (n = 32), obviously abnormal (n = 33), or having subtle abnormal findings (n = 35) by 2 senior radiologists who reached a consensus for each individual case. Subsequently, 5 raters with varying degrees of experience independently viewed and interpreted the cases as normal or abnormal. Raters viewed each image 1 month apart on a PACS and on the Web-based PC imaging system. McNemar tests were used to compare accuracy of interpretation across both imaging systems. Confidence intervals (CIs) were calculated for differences in the proportion assessed incorrectly on the PACS, compared with the Web-based PC imaging system. RESULTS: There was no relation between accuracy of detection and the system used to evaluate X-ray images (P = 0.92). The total percentage of incorrect interpretations on the Web-based PC imaging system was 23.2%, compared with 23.6% on the PACS (P = 0.92). For all raters combined, the overall difference in proportion assessed incorrectly on the PACS, compared with the PC system, was not significant at 0.4% (95%CI, -3.5% to 4.3%). CONCLUSION: The high-resolution Web-based imaging system via PC is an adequate alternative to a PACS clinical workstation. Accordingly, the provision of a more extensive network of workstations throughout the hospital setting could have potentially significant cost savings.     

Liver and bone window settings for soft-copy interpretation of chest and abdominal CT

OBJECTIVE: We evaluated whether the use of multiple window and level settings on a soft-copy workstation improves diagnostic accuracy on chest and abdominal CT. We hypothesized that routinely using window and level settings during soft-copy interpretation would beneficially affect the final diagnosis without compromising efficiency. MATERIALS AND METHODS: Two hundred three randomly selected abdominal and chest CT scans were interpreted by three radiologists using a four-monitor soft-copy workstation (images per screen, nine; resolution, 2K). After the initial interpretations, all scans were reevaluated by the same radiologists using additional liver and bone window and level settings. Differences in conspicuity and characterization of abnormalities were graded on a three-point scale. RESULTS: Conspicuity and characterization of abnormalities were improved in 67% of abnormal findings (81/121; p = 0.01). Improvement (a finding that substantially affected the final diagnosis) was present in 18% of abnormal findings (22/121; p = 0.04). On average, the evaluation of images at multiple window and level settings required an additional 40 sec per case. CONCLUSION: The use of multiple window and level settings during soft-copy interpretation resulted in improved lesion detectability and characterization with greater diagnostic efficacy. Using soft-copy workstations, radiologists can evaluate images using multiple settings without compromising efficiency.     

Hard-copy versus soft-copy image reading for detection of ureteral stones on abdominal radiography

PURPOSE: The purpose of this study was to compare the usefulness of soft-copy images displayed on a cathode ray tube (CRT) with hard-copy images (film images) for detecting ureteral stones on abdominal radiography. MATERIALS AND METHODS: Five radiologists read images from 50 cases of ureteral stones and 50 normal cases diagnosed on the basis of intravenous urography and CT. For hard-copy reading, 10-bit images at 3,520x4,280 pixels obtained by computed radiography were printed on 14x17-inch films. For soft-copy reading, 8-bit images were displayed on a 17-inch monochrome monitor at 1,024x1,280 pixels. The study items were area under receiver operating characteristics (ROC) curve (Az), ureteral stone detection sensitivity and specificity, and reading time. RESULTS: For soft-copy and hard-copy images, the average Az values were 0.855 and 0.851, sensitivity was 62.8% and 62%, and specificity was 70.8% and 62.4%, respectively. There were no statistically significant differences between these values. Reading time was 106.6 min for soft-copy images, significantly longer than the 71.2 min for hard-copy images (p<0.05). CONCLUSION: Although soft-copy image reading time was longer than hard-copy image reading time, the ability to diagnose ureteral stones on abdominal radiography did not differ for soft- and hard-copy images.