Interobserver variation in the detection of pulmonary venous hypertension in chest radiographs

A total of 171 sets of chest radiographs chosen randomly were reviewed independently by two residents in their second year of training, and two experienced radiologists. The degree of interobserver agreement in an overall assessment of pulmonary venous hypertension and in the assessment of five signs, indicative of pulmonary venous hypertension was determined by kappa statistics. The average level of agreement was moderate (0.51-0.56) for the overall assessment and the signs of perivascular clouding, perihilar haze and pleural effusion, and poorer (0.31-0.38) for flow shift and Kerley lines. In the overall assessment agreement between experienced radiologists was slightly better (0.63) than average. Multiple reader interpretation is recommended for the assessment of low-grade pulmonary venous hypertension.     

Inter-observer variation in the interpretation of chest radiographs for pneumonia in community-acquired lower respiratory tract infections

AIM: To assess inter-observer variation in the interpretation of chest radiographs of individuals with pneumonia versus those without pneumonia. MATERIALS AND METHODS: Chest radiographs of out-patients with a lower respiratory tract infection (LRTI) were assessed for the presence of infiltrates by radiologists from three local hospitals and were reassessed by one university hospital radiologist. Various measures of inter-observer agreement were calculated. RESULTS: The observed proportional agreement was 218 in 243 patients (89.7%). Kappa was 0.53 (moderate agreement) with a 95% confidence interval of 0.37 to 0.69. The observed positive agreement (59%) was much lower than for negative agreement (94%). Kappa was considerably lower, if chronic obstructive pulmonary disease was present (kappa = 0.20) or Streptococcus pneumoniae (kappa = -0.29) was the infective agent. CONCLUSION: The overall inter-observer agreement adjusted for chance was moderate. Inter-observer agreement in cases with pneumonia was much worse than the agreement in negative (i.e. non-pneumonia) cases. A general practitioner's selection of patients with a higher chance of having pneumonia for chest radiography would thus not improve the observer agreement.     

Evaluation of competence in the interpretation of chest radiographs

RATIONALE AND OBJECTIVES: The purpose of this study was to determine relative rates of missed diagnoses for radiologists as a measure of competence in interpreting chest radiographs. MATERIALS AND METHODS: Cases involving differing interpretations of chest radiographs were collected from January 1994 through December 1999 by faculty (chest and nonchest radiology specialists) in an academic radiology department. A quarterly peer-review process designated cases months after the fact, and anonymously, as no miss or as class I (nondiagnosable), class II (very difficult diagnosis), class III (should be diagnosed most of time), or class IV (should almost always be diagnosed) missed diagnoses. The rates and classes of missed diagnoses were compared among chest faculty and for the nonchest radiology specialists as a group. RESULTS: Chest radiologists read 184,977 studies, and nonchest radiologists read 300,684 studies. Of these, 243 missed diagnoses were classified (classes I and II, 184 cases; class III, 50; and class IV, nine). No difference was detected in the rate of class III and IV misses among chest faculty, but nonchest faculty had significantly more class III (P = .022) and class IV misses (P = .016). CONCLUSION: Random sampling of differing interpretations can yield a relative rate of missed diagnoses for radiologists. No difference was detected in clinically important misses (ie, classes III and IV) among chest radiologists, but a statistically significantly higher rate of seemingly obvious misdiagnoses was found for nonchest specialty radiologists. Potential biases may have influenced this analysis, including disease prevalence, sampling, clinical factors, observer variability, and truth-in-diagnosis.     

Interobserver variation in reporting CT arthrograms of the shoulder

Computed tomography (CT) arthrography of the shoulder is an imaging modality of great diagnostic accuracy with regard to glenohumeral instability and in particular labral lesions. Interpretation of the scans is made difficult by the frequent occurrence of normal anatomic variants and the complexity of injuries to the bone and soft tissues. We selected a continuous sample of 50 CT arthrograms of the shoulder and they were reported by two consultant musculoskeletal radiologists. The results were collated and analysed for the level of agreement. Hill-Sachs showed Kappa (K) statistic to be 0.37 (fair agreement), soft tissue Bankart 0.32 (fair agreement), bony Bankart 0.61 (substantial agreement), anterior capsular laxity 0.41 (moderate agreement) and glenohumeral osteoarthritis 0.20 (slight agreement). All the results were significant with a p value of <0.05. Nine (18%) of the 50 scans were in complete agreement. The results demonstrate that there can be considerable interobserver variation (IOV) in the reports of a CT arthrogram of a shoulder. They highlight the potential difficulties in reporting such images and suggests ways in which the report could be more focussed to provide a clinically reliable report and one which matches the surgical findings accurately.     

Interobserver variability in the interpretation of renal digital subtraction angiography.

OBJECTIVE: Our purpose was to analyze interobserver variability in the interpretation of renal digital subtraction angiography and to describe the main factors associated with observer discrepancies. MATERIALS AND METHODS: Forty-nine cases of unilateral atheromatous renal artery stenosis of more than 60% were quantified first by local investigators in a multicenter study and then by five other radiologists. Differences between radiologists for the minimum diameter (Dmin), the reference diameter (Dref), and the percentage of stenosis of the renal arteries were analyzed. Interpretations by the local investigators were then compared with the gold standard, defined as the mean for the five radiologists. RESULTS: The average SD for estimation of all renal artery stenoses by all radiologists was 7% for stenosis percentage, 0.5 mm for Dmin, and 0.7 mm for Dref. Main discrepancies occurred more frequently in cases of weakly opacified renal artery stenosis and poststenotic dilatation. The observations of local investigators disagreed by more than two SDs (14%) with the gold standard for 11 of 49 cases (22%). CONCLUSION: The accuracy of digital subtraction angiography in renal artery interpretations is poor because of variations in evaluating both Dmin and Dref. Precise and reproducible methods for quantification of renal artery stenosis are required.     

Interobserver variation in the radiographic diagnosis of duodenal ulcer disease

Interobserver variation was estimated in radiologists' detection of duodenal ulcer and deformity of the duodenal bulb. A consecutive series of 156 patients with upper abdominal pain had a double contrast barium examination. The films were first read by routine by a specialist in radiology. A second reading was performed by another specialist who did not know the result of the first reading. Overall agreement between the two readings in the detection of duodenal ulcer and deformity of the duodenal bulb was 0.91 and 0.94, respectively. The overall agreement was adjusted for the expected chance agreement and kappa values were calculated. Kappa was 0.63 for the detection of duodenal ulcer and 0.85 for the detection of deformity of the duodenal bulb.     

Interobserver delineation variation in lung tumour stereotacticbody radiotherapy

Objectives In radiotherapy, delineation uncertainties are important as they contribute to systematic errors and can lead to geographical miss of the target. For margin computation, standard deviations (SDs) of all uncertainties must be included as SDs. The aim of this study was to quantify the interobserver delineation variation for stereotactic body radiotherapy (SBRT) of peripheral lung tumours using a cross-sectional study design. Methods 22 consecutive patients with 26 tumours were included. Positron emission tomography/CT scans were acquired for planning of SBRT. Three oncologists and three radiologists independently delineated the gross tumour volume. The interobserver variation was calculated as a mean of multiple SDs of distances to a reference contour, and calculated for the transversal plane (SD(trans)) and craniocaudal (CC) direction (SD(cc)) separately. Concordance indexes and volume deviations were also calculated. Results Median tumour volume was 13.0 cm(3), ranging from 0.3 to 60.4 cm(3). The mean SD(trans) was 0.15 cm (SD 0.08 cm) and the overall mean SD(cc) was 0.26 cm (SD 0.15 cm). Tumours with pleural contact had a significantly larger SD(trans) than tumours surrounded by lung tissue. Conclusions The interobserver delineation variation was very small in this systematic cross-sectional analysis, although significantly larger in the CC direction than in the transversal plane, stressing that anisotropic margins should be applied. This study is the first to make a systematic cross-sectional analysis of delineation variation for peripheral lung tumours referred for SBRT, establishing the evidence that interobserver variation is very small for these tumours.     

The role of clinical history in the interpretation of chest radiographs

ObjectiveThis review will appraise the literature pertaining to the influences that clinical history has on the action of assessing the chest radiograph.Key findingsThere remains conflicting evidence on the impact of clinical history on chest radiography. Some research suggests that clinical history has the potential to influence the reporter in a negative way by limiting their search strategy to a more focussed search. Image interpretation is more accurate when reporters are allowed to conduct a free search of the chest image, untainted by preconceived concepts.ConclusionClinical history needs to be accessed appropriately to aid and not stifle accurate image interpretation. Reporters need to be aware of the potential bias clinical history can introduce to their reporting and develop strategies to alleviate this as much as possible.Implications for practiceA greater understanding of the potential bias of clinical history on the process of image interpretation is required by all reporters. Reporters need to develop an approach and strategy when accessing clinical history. Novice reporters need to be educated regarding the impact of clinical history on their reporting.     

Visual-spatial ability and interpretation of three-dimensional information in radiographs

OBJECTIVES: To investigate whether skill in the interpretation of three-dimensional (3D) information in radiographs utilizing the parallax phenomenon is associated with visual-spatial ability and whether development of this skill is related to visual-spatial ability. METHODS: Eighty-six individuals with a median age of 25 years participated in the study. It was organized into three parts: (1) assessment before training, (2) training in object depth localization utilizing parallax and (3) assessment after training. Before training, visual-spatial ability was assessed with a mental rotation test, MRT-A; skill in interpreting 3D information was assessed with two specifically designed proficiency tests: a radiography test, which assessed the ability to interpret 3D information in radiographs utilizing motion parallax and a principle test which assessed understanding of the principles of motion parallax. After training, skill in interpreting 3D information was reassessed. Improvement was defined as the difference between test scores after training and before training. Multiple linear regression was used to analyse the effect of student and training characteristics on proficiency test results and improvement. RESULTS: Radiography test results after training and improvement of radiography test results were significantly associated with MRT-A scores (P<0.001 and P=0.020, respectively). Principle test results were high before training and did not improve after training. The test results were associated with MRT-A both before (P=0.009) and after training (P=0.003). CONCLUSIONS: Understanding of the parallax phenomenon is associated with visual-spatial ability. Development of the skill to interpret 3D information in radiographs utilizing parallax is facilitated for individuals with high visual-spatial ability.     

Interobserver variation in the diagnosis of bronchiectasis on high-resolution computed tomography

The purpose of our study was to determine interobserver variation in the analysis of high-resolution computed tomography (HRCT) in the lungs of patients with clinically suspected bronchiectasis. HRCT scans of 88 patients were analysed independently by three radiologists with variable experience in thoracic radiology using a subjective scoring system to record bronchi as normal, mildly abnormal or severely abnormal. The presence, severity and distribution of bronchial dilatation and bronchial wall thickening were recorded. Kappa values were calculated for assessment of interobserver agreement. Agreement between the three readers was good for the detection of bronchiectasis (kappa 0.78) and assessment of its severity (0.68), detection of bronchial wall thickening (0.64) and moderately good for the assessment of its severity (0.58) on a per-patient basis. When individual lobes were analysed, agreement was moderately good for the detection of abnormal bronchi (0.59). Agreement on the extent of abnormal bronchi using five categories was only fair (0.39), but was good when differences of one category were ignored (0.63). Interobserver variation with HRCT in suspected bronchiectasis appears satisfactory for comparative studies.     

Acute abdominal pain in children: usefulness of three-view abdominal radiographs in the emergency department

Diagnostic imaging is often an integral component in the workup of a pediatric patient with acute abdominal pain. The purpose of this study was to compare the diagnostic value of a three-view acute abdominal series (AAS) with that of a single supine view (SSV) in children with acute abdominal pain. All subjects aged ≤18 years that underwent an emergency three-view AAS examination for acute abdominal pain at a single urban hospital system were included. Retrospective evaluation of radiological diagnosis, number of radiological images, further imaging, management, and clinical outcomes was performed. “Positive” AAS studies were compared with corresponding SSV images for direct comparison of diagnostic value. Standard nonparametric statistical evaluation was performed. Five hundred forty-one AAS studies were included in the study. Greater than three radiographs were acquired in 29 % (153/541) of the subjects. Two hundred ninety-nine out of 541 AAS studies included a technically adequate SSV of the abdomen and pelvis. Most AAS examinations were categorized as negative (n = 485; 90 %). Of the 56 examinations initially classified as positive, there was no significant statistical difference between diagnostic accuracy between the AAS and SSV on retrospective evaluation. For pediatric subjects with nontraumatic acute abdominal pain, the yield of conventional radiographic study is exceedingly low. If required, a technically adequate single supine anteroposterior (AP) view of the abdomen and pelvis is sufficient for initial radiographic evaluation while reducing unnecessary radiation exposure to the patient.     

Diagnosis of pneumoperitoneum on supine abdominal radiographs

A blinded, retrospective study was performed to determine the value of supine abdominal radiographs in diagnosing pneumoperitoneum. Supine films from 44 cases of pneumoperitoneum were randomly interspersed among supine films from 87 control subjects without free air, and the films were reviewed for the presence or absence of various signs of pneumoperitoneum, including Rigler's sign (gas on both sides of the bowel wall), the falciform ligament sign (gas outlining the falciform ligament), the football sign (gas outlining the peritoneal cavity), the inverted-V sign (gas outlining the medial umbilical folds), and the right-upper-quadrant gas sign (localized gas in the right upper quadrant). One or more of these signs were present in 26 cases (59%) of pneumoperitoneum, including the right-upper-quadrant gas sign in 18 cases (41%), Rigler's sign in 14 cases (32%), and the falciform ligament and football signs in one case each (2%). Unfortunately, there were frequent errors in the interpretation of the right-upper-quadrant gas sign and Rigler's sign, with a total of 11 false-positive cases (13%). Further analysis of the true-positive right-upper-quadrant gas signs showed that these gas collections were always triangular or linear with an inferolateral to superomedial orientation and, if triangular, a concave superolateral border. In the true-positive Rigler's signs, the bowel wall thickness ranged from 1 to 8 mm, whereas the false positives all had a bowel wall thickness of 1 mm or less. Proper interpretation of the various signs of pneumoperitoneum on supine films should lead to more accurate diagnosis of this condition.