Observer Agreement of Mammographic Interpretation

Abstract: The purpose of this study was to determine the inter- and intraobserver agreement of mammographic interpretation among three board-certified radiologists with different levels of expertise in mammography who interpret mammograms, and to use the data to improve quality assurance of mammography. Four hundred and ninety-five randomly selected mammograms from a pool of 9, 326 mammograms performed between 1987 and 1990 were reexamined by three board certified radiologists without prior knowledge of the original interpretations. Each radiologist was asked to assign the mammogram to any of the three categories of normal, indeterminate, and suspicious for malignancy, and also to provide recommendations for further workup (breast physical examination, short-term follow-up mammography, ultrasound, etc.). In addition, 120 mammograms were reinterpreted for the second time by the same radiologist in order to determine the intraobserver variability. Results were also compared with the original readings. Kappa statistics was used in the analysis of the agreement. The interobserver agreement for the overall diagnosis was 83.6% between radiologists #1 and #2, 86.5% between #1 and #3, and 81% between radiologist #2 and #3. The intraobserver agreement was 98.7%, 89.1%, and 100% for radiologists #1, #2, and #3, respectively. The overall agreement between the three radiologists for the interpretation of 495 mammograms was good to excellent. Some disagreement was seen in detecting suspicious calcifications, short-term follow-up recommendations, and further diagnostic workup. The study demonstrated that the overall reading of mammograms by each radiologist was significantly different from the original interpretation (p < 002). The p-value indicates that the agreement was significantly greater than would be expected by chance alone.?     

Observer Variability of the Breast Imaging Reporting and Data System (BI-RADS) Lexicon for Mammography

AIM: We aimed to determine the inter- and intra-observer variabilities between breast radiologists and a general radiologist in categorizing mammographic lesions using the Breast Imaging Reporting and Data System (BI-RADS), and to evaluate the effects of the histopathologic results on the variability. METHODS: Mammograms from 142 women who underwent biopsy were evaluated. 3 breast radiologists (2 with >10 years experience and 1 with 1 year experience) and 1 general radiologist retrospectively reviewed mammograms twice within an 8-week interval. Inter- and intra-observer variabilities were assessed with Cohen's kappa statistic, and the positive predictive value for final assessments was calculated. RESULTS: The intra-observer variability for mass and calcification assessments was moderate to almost perfect (kappa values: 0.41-1) for breast imagers and was fair to substantial for the general radiologist (kappa values: 0.21-0.8). Inter-observer agreement between the breast imagers was higher than between the breast and general radiologists. There was no apparent difference in agreement between observers for malignant and benign subgroups. CONCLUSIONS: The differences in intra- and inter-observer agreement between the breast imagers and the general radiologist affirm the utility of the BI-RADS lexicon. The histopathologic results of the lesions do not affect the agreement. BI-RADS is a simple and adequate tool for assessing mammograms, even after only 1 year of training.     

Mammography: interobserver variability in breast density assessment

Our objective was to determine the interobserver variability of breast density assessment according to the Breast Imaging Reporting and Data System (BI-RADS) and to examine potential associations between breast density and risk factors for breast cancer. Four experienced breast radiologists received instructions regarding the use of BI-RADS and they assessed 57 mammograms into BI-RADS density categories of 1-4. The weighted kappa values for breast density between pairs of observers were 0.84 (A, B) (almost perfect agreement); 0.75 (A, C), 0.74 (A, D), 0.71 (B, C), 0.77 (B, D), 0.65 (C, D) (substantial agreement). The weighted overall kappa, measured by the intraclass correlation coefficient (ICC), was 0.77 (95% CI: 0.69-0.85). Body mass index was inversely associated with high breast density. In conclusion, overall interobserver agreement in mammographic interpretation of breast density is substantial and therefore, the BI-RADS classification for breast density is useful for standardization in a multicentre study.     

Which breast to biopsy: an expanding dilemma.

To provide insight into the significance of findings reported from screening asymptomatic women for breast cancer, we reviewed 19,928 mammographic studies with the accompanying physical examination and correlated these findings with 554 breast biopsies. Of 83 breast biopsies with suspicious findings on both physical examination and mammography, 72 demonstrated breast cancer (87%). Of 155 biopsies for suspicious changes on mammography alone, 50 (32%) demonstrated breast cancer. The accuracy of suspicious mammography was independent of findings limited to physical examination, 34 (17%) had breast cancer; 31 were in 152 biopsies of patients with mammography interpreted as normal (20%) and three were in biopsies of 52 patients (6%) in whom a visualized mass was interpreted as benign. One hundred and twelve breast biopsies were performed for changes interpreted as normal or benign. Six malignancies were discovered (5%). No cancer was found in 31 biopsies for nonpalpable benign mammographic abnormalities. Our results emphasize the importance of discriminating between nonvisualization of a mass and mammographic recognition of either a benign or malignant tumor. The reliability of interpretation is considerably greater for a visualized lesion than a nonvisualized one.     

Early experience with needle localization and biopsy of mammographic lesions

Increasing awareness of the value of mammography by both physicians and the public has resulted in women presenting more commonly with impalpable breast lesions. This study reviews the radiology and pathology of 58 such lesions biopsied by the Monash Medical Centre Breast Unit between August 1987 and October 1988. Abnormal mammograms were reported by one of two independent radiologists in the normal course of practice and placed into one of five categories according to the radiological appearance. Those lesions scoring greater than or equal to 3 were then needle localized, excised and examined histologically. All mammograms were later reported as unknowns by the other radiologist and similarly scored. Sixteen (28%) of these lesions were invasive or in situ carcinoma and of these a significant number were scored differently by the two radiologists. The results indicate that needle localization biopsy of suspicious mammographic lesions is a safe, accurate method for the diagnosis of early breast cancer. The results also show significant variation between radiologists and demonstrate the need for double reading of screening mammographic films. It is suggested that doubtful lesions require more extensive work-up with compression/magnification and other special views, the aim being more accurate radiological assessment and a reduction in the benign biopsy rate.     

Misclassification of Breast Imaging Reporting and Data System (BI‐RADS) Mammographic Density and Implications for Breast Density Reporting Legislation

USA states have begun legislating mammographic breast density reporting to women, requiring that women undergoing screening mammography who have dense breast tissue (Breast Imaging Reporting and Data System [BI‐RADS] density c or d) receive written notification of their breast density; however, the impact that misclassification of breast density will have on this reporting remains unclear. The aim of this study was to assess reproducibility of the four‐category BI‐RADS density measure and examine its relationship with a continuous measure of percent density. We enrolled 19 radiologists, experienced in breast imaging, from a single integrated health care system. Radiologists interpreted 341 screening mammograms at two points in time 6 months apart. We assessed intra‐ and interobserver agreement in radiologists'; interpretations of BI‐RADS density and explored whether agreement depended upon radiologist characteristics. We examined the relationship between BI‐RADS density and percent density in a subset of 282 examinations. Intraradiologist agreement was moderate to substantial, with kappa varying across radiologists from 0.50 to 0.81 (mean = 0.69, 95% CI [0.63, 0.73]). Intraradiologist agreement was higher for radiologists with ≥10 years experience interpreting mammograms (difference in mean kappa = 0.10, 95% CI [0.01, 0.24]). Interradiologist agreement varied widely across radiologist pairs from slight to substantial, with kappa ranging from 0.02 to 0.72 (mean = 0.46, 95% CI [0.36, 0.55]). Of 145 examinations interpreted as “nondense” (BI‐RADS density a or b) by the majority of radiologists, 82.8% were interpreted as “dense” (BI‐RADS density c or d) by at least one radiologist. Of 187 examinations interpreted as “dense” by the majority of radiologists, 47.1% were interpreted as “nondense” by at least one radiologist. While the examinations of almost half of the women in our study were interpreted clinically as having BI‐RADS density c or d, only about 10% of examinations had percent density >50%. Our results suggest that breast density reporting based on a single BI‐RADS density interpretation may be misleading due to high interradiologist variability and a lack of correspondence between BI‐RADS density and percent density.     

Breast saving therapy: diagnostic problems in after care

From 1.1.1977 through 31.12.1985 two hundred seventy-five patients with stages pTis, pT1-2, pN0-1, cM0 breast cancer were treated by breast-conserving therapy. Six patients (2%) were lost to follow-up. Fifty-four patients developed significant subcutaneous or parenchymal induration in the treated breast. The induration was considered clinically suspicious for recurrence in 30/54 and non-suspicious in 24/54. Of the 30 suspicious indurations mammography was diagnostic or suggestive of recurrence in 12 (40%), and the diagnosis of recurrence could be confirmed microscopically in 16 (53%). Mammography showed no pathologic findings in 14/30 (42%) patients with suspicious indurations, and in 12/30 (40%) the impression of benignity was also confirmed microscopically. In patients whose mammograms were diagnostic or suggestive of tumor, malignancy was demonstrated by pathologic examination in all cases. In 21/24 (87.5%) clinically benign indurations no tumor could be found on microscopic examination. Twenty of these patients had a normal mammogram, and in only 1 of these was a recurrence discovered incidentally during a cosmetic operation. Four of the 24 patients with clinically benign indurations had pathologic mammograms, and in 2 of these the diagnosis of recurrence was established histologically. - The rate of agreement between suspicious clinical and mammographic or microscopic findings was about 50%, whereas the correlation for clinically benign indurations was greater than 80%.     

Outcome of men presenting with clinical breast problems: the role of mammography and ultrasound

The purpose of this study was to determine the outcome of men presenting with clinical breast problems for breast imaging and to evaluate the role of mammography and ultrasound in the diagnosis of benign and malignant breast problems. We retrospectively reviewed clinical, radiographic, and pathologic records of 165 consecutive symptomatic men presenting to Breast Imaging over a 4 year period. We assessed the clinical indication for referral, mammographic findings, sonographic findings, histologic results, and clinical outcomes. Patients ranged in age from 22 to 96 years. Breast Imaging Reporting and Data System (BI-RADS) category 4 and 5 mammograms and solid sonographic masses were considered suspicious for malignancy. Six of 165 men (4%) had primary breast carcinoma, which were mammographically suspicious in all 6 (100%). Five were invasive ductal carcinoma and one was ductal carcinoma in situ (DCIS). Of 164 mammograms, 20 (12%) were suspicious. Six were cancer and 14 were benign. Clinical follow-up for 2 years or biopsy results were available for 138 of the 165 men (84%). Twelve with benign mammographic findings had benign biopsies. All men with benign mammography not undergoing biopsy were cancer free. Sensitivity for cancer detection (mammography) was 100% and specificity was 90%. Positive predictive value (mammography) was 32% (6 of 19) and the negative predictive value was 100%. Sonography was performed in 68 of the 165 men (41%). Three of three cancers (100%) were solid sonographic masses. There were 9 of 68 false-positive examinations (13%). Sensitivity and negative predictive value for cancer detection (ultrasound) was 100% and specificity was 74%. The most common clinical indication for referral was mass/thickening (56%). Mammography had excellent sensitivity and specificity for breast cancer detection and should be included as the initial imaging examination of men with clinical breast problems. The negative predictive value of 100% for mammography suggests that mammograms read as normal or negative need no further examination if the clinical findings are not suspicious. A normal ultrasound in these men confirms the negative predictive value of a normal mammogram.     

The diagnostic accuracy of mammography in the evaluation of male breast disease.

BACKGROUND: The role of mammography in the evaluation of male patients presenting with breast disease is controversial. This controversy is a function of the lack of specific data concerning the diagnostic accuracy of mammography when used in this clinical setting. The purpose of this study was to define the diagnostic accuracy of mammography in the evaluation of male breast disease. METHODS: One hundred and four prebiopsy mammograms from 100 patients with tissue diagnoses were read blindly by two independent radiologists, and placed into one of five predetermined categories: definitely malignant, possibly malignant, gynecomastia, benign mass, and normal. Radiologic/pathologic correlation was performed and the sensitivity (Sn), specificity (Sp), positive (Ppv) and negative predictive value (Npv), and accuracy (Ac) for each of the mammographic diagnostic category determined. RESULTS: The pathologic diagnoses were 12 cancers, including 1 patient with bilateral breast cancer, 70 cases of gynecomastia, 16 benign masses, and 6 normals. The accuracy data for the mammographic diagnostic categories are as follows: malignant (combined definitely and possibly malignant), Sn 92%, Sp 90%, Ppv 55%, Npv 99%, Ac 90%; and overall benignity (combined gynecomastia, benign mass, and normal), Sn 90%, Sp 92%, Ppv 99%, Npv 55%, Ac 90%. Six cancers (50%) coexisted with gynecomastia. CONCLUSIONS: Mammography can accurately distinguish between malignant and benign male breast disease. Although not a replacement for clinical examination, its routine use could substantially reduce the need for biopsy in patients whose mammograms and clinical examination suggest benign disease.     

Biopsy of nonpalpable breast lesions

One hundred-three needle-direct breast biopsies were performed in 99 patients. Nine cancers were detected. All of the patients but one had in situ tumor or stage I disease. After analysis, the mammograms were submitted to radiologists for restudy in a blinded fashion. On restudy, 50 percent of the malignancies were read as low suspicion or benign whereas the other 50 percent were read as malignant. Variation between interpretations was significant, with complete agreement among the radiologists in 16 percent of cases. We concluded that needle-directed biopsy for occult breast lesions is very effective in bringing patients with a more favorable prognosis to early surgery. The false-positive rate will be high and the interpretation of a given mammogram will vary, depending on the radiologist. Mammography for subtle lesions is sensitive but nonspecific; therefore, it is wise to biopsy all lesions that are even slightly suspicious, particularly in patients with a previous history of breast cancer or other risk factors.     

The impact of mammography on breast biopsy.

To determine the efficacy of mammography in the detection of early breast carcinoma at an urban teaching hospital, the results of all breast biopsies performed between 1983 and 1987 that were preceded by mammographic examination were retrospectively reviewed. There were 503 women in this population. Malignancy was detected in 79 cases (15.7%); 21 were in situ and 58 were invasive. Among all nonpalpable malignancies, 53.0 per cent were in situ, while only 2.4 per cent of all palpable malignancies were in situ. An abnormality was found in 374 mammograms (74%), and 73 (19.5%) were malignant. The abnormality most likely to represent a malignancy (44% yield) was spiculated density, followed by clustered microcalcifications (25%), mass (22%), and asymmetric density (14%). Six malignancies were detected by biopsy for clinical indications, despite a negative mammogram (4.7% false- negative rate). The interpretation of mammograms by radiologists carried a 2.4 per cent false-negative rate. The mammographic features of mass, clustered microcalcifications, spiculations or asymmetric density should generally mandate breast biopsy, although the clinical examination should remain an important basis for management decisions. An aggressive approach toward screening mammography and breast biopsy based on mammographic criteria may enhance survival among women with breast carcinoma.     

Reproducibility of BI-RADS breast density measures among community radiologists: a prospective cohort study

Using data from the Vermont Breast Cancer Surveillance System (VBCSS), we studied the reproducibility of Breast Imaging Reporting and Data System (BI-RADS) breast density among community radiologists interpreting mammograms in a cohort of 11,755 postmenopausal women. Radiologists interpreting two or more film-screen screening or bilateral diagnostic mammograms for the same woman within a 3- to 24-month period during 1996-2006 were eligible. We observed moderate-to-substantial overall intra-rater agreement for use of BI-RADS breast density in clinical practice, with an overall intra-radiologist percent agreement of 77.2% (95% confidence interval (CI), 74.5-79.5%), an overall simple kappa of 0.58 (95% CI, 0.55-0.61), and an overall weighted kappa of 0.70 (95% CI, 0.68-0.73). Agreement exhibited by individual radiologists varied widely, with intra-radiologist percent agreement ranging from 62.1% to 87.4% and simple kappa ranging from 0.19 to 0.69 across individual radiologists. Our findings underscore the need for additional evaluation of the BI-RADS breast density categorization system in clinical practice.     

Nonpalpable breast lesions at biopsy. A detailed analysis of radiographic features.

Several studies have demonstrated that mammographic screening of asymptomatic women results in a lower mortality rate where breast cancer is concerned. Often, screening mammograms reveal a nonpalpable radiographic abnormality and the diagnosis must be determined by an excisional biopsy after radiographic needle localization. The mammographic features associated with 179 nonpalpable breast abnormalities biopsied after radiographic needle localization were carefully characterized. There were 41 carcinomas (23%) in the series. The aim of this study was to determine which radiographic findings, if any, strongly portend the presence of either a malignant or benign lesion. Mammographic features that were commonly associated with malignancy include a change from a previous mammogram, a distortion of the surrounding architecture, the association of a soft tissue density and calcifications, and the presence of more than ten calcifications in the lesion. The radiographic abnormalities which were more commonly associated with benign disease include well-defined densities without calcifications, asymmetric densities without calcifications, and abnormalities consisting solely of a focus of mammographic calcifications that have fewer than ten concretions. The incidence of malignancy in lesions having these mammographic characteristics was only 5.5%. On the basis of these results alone, no firm threshold for biopsy can be recommended. The risks of deferring biopsy until there is worsening of the mammographic image remains to be determined.     

Abnormal mammographic findings. A critical appraisal

We performed a 1 year review of mammographic interpretations and breast biopsy experiences at Madigan Army Medical Center, correlating biopsy results with mammographic interpretations when possible. Fibrocystic mammary dysplasia was reported in two thirds of our patients, with interpretations heavily weighted toward findings of moderate or severe dysplasia. Sixteen percent of our patients had indeterminate or suspicious mammograms, and only 11 percent of the mammograms were read as normal. Of 19 patients who underwent needle-directed biopsy for nonpalpable lesions and suspicious or indeterminate mammograms, two had invasive cancer and one lobular carcinoma in situ. Our data suggest that many patients who have indeterminate mammograms are followed in preference to early biopsy if there is no palpable lesion and no high yield radiologic criteria of malignancy.     

Validation of Different Techniques in Physical Examination of Breast

The majority of patients attending breast clinics are found to be suffering from benign conditions. The detailed investigations of every patient would add to the cost of care and burden the laboratories. A detailed clinical evaluation might limit the use of thorough investigations for suspicious lesions only. This cross sectional study involved the patients with various benign and malignant conditions of breast, who attended outpatient clinic and surgical ward at All India Institute of Medical Sciences, New Delhi from June 2009 to May 2011. The study started with a training of the resident (observer 2) in various breast examination techniques by a professor of surgery (observer 1), who was well trained in the discipline of breast surgery by internationally renowned breast experts. The different techniques of breast examination were validated after calculation of intra and inter-observer variation. Excellent agreement was observed between both the observers. The diagnostic accuracy ratio for most variables ranges from 0.9 to 1. The dimpling of skin on inspection had a low kappa (coefficient of agreement?=?0.48) and consistency of lymph node on palpation had a kappa 0.38. All other variables showed high agreement. The present study was successful in training the resident and validating the different techniques in physical examination of breast.     

HRCT diagnosis of diffuse parenchymal lung disease: inter-observer variation

BACKGROUND: This study was designed to measure inter-observer variation between thoracic radiologists in the diagnosis of diffuse parenchymal lung disease (DPLD) using high resolution computed tomography (HRCT) and to identify areas of difficulty where expertise, in the form of national panels, would be of particular value. METHODS: HRCT images of 131 patients with DPLD (from a tertiary referral hospital (n = 66) and regional teaching centres (n = 65)) were reviewed by 11 thoracic radiologists. Inter-observer variation for the first choice diagnosis was quantified using the unadjusted kappa coefficient of agreement. Observers stated differential diagnoses and assigned a percentage likelihood to each. A weighted kappa was calculated for the likelihood of each of the six most frequently diagnosed disease entities. RESULTS: Observer agreement on the first choice diagnosis was moderate for the entire cohort (kappa = 0.48) and was higher for cases from regional centres (kappa = 0.60) than for cases from the tertiary referral centre (kappa = 0.34). 62% of cases from regional teaching centres were diagnosed with high confidence and good observer agreement (kappa = 0.77). Non-specific interstitial pneumonia (NSIP) was in the differential diagnosis in most disagreements (55%). Weighted kappa values quantifying the likelihood of specific diseases were moderate to good (mean 0.57, range 0.49-0.70). CONCLUSION: There is good agreement between thoracic radiologists for the HRCT diagnosis of DPLD encountered in regional teaching centres. However, cases diagnosed with low confidence, particularly where NSIP is considered as a differential diagnosis, may benefit from the expertise of a reference panel.     

膝关节骨关节炎全关节磁共振成像评分观察者间一致性评价

目的 探讨应用全关节磁共振成像评分(WORMS)系统对膝关节各结构特征进行半定量评价的观察者间一致性以及观察者临床阅片经验对评分结果的影响.方法 利用"祛风止痛胶囊对膝骨关节炎的临床作用及机制研究"临床研究项目数据库,随机抽取2018年5至10月摄取的膝关节骨关节炎患者的磁共振像(MRI)10例,分别由3位年资不同、拥...     

Intraobserver Agreement on Histopathologic Evaluations of Core Breast Biopsies

The number of performed core biopsies of the breast as diagnostic workup is increasing in many European countries. We measured the intraobserver variability in pathological assessment of breast core biopsies. Furthermore, we studied potential modifiers of agreement between the assessments. Two hundred and fifty‐six breast biopsies were evaluated twice in a blinded fashion by two pathologists. We calculated the observed and the chance‐corrected (weighted) intraobserver agreement (kappa) using the B‐categorization scheme (B1: normal or not interpretable, B2: benign, B3: benign but of uncertain biological potential, B4: suspicious of malignancy, B5: malignant). The observed agreement between the first and the second assessments were 0.80 (95% CI: 0.75–0.85) for pathologist 1 and 0.81 (95% CI: 0.76–0.86) for pathologist 2. The chance‐corrected agreements were 0.85 (95% CI: 0.80–0.89) and 0.81 (95% CI: 0.76–0.87), respectively. The most frequent disagreement was between B1 and B2 for pathologist 1 (N = 34 out of 50 disagreements, 68%) and between B2 and B3 for pathologist 2 (N = 23 out of 48 disagreements, 48%). Our study shows that the chance‐corrected agreement between the histopathological evaluations of breast biopsies based on the B‐categorization scheme is almost perfect. The level of agreement is modified by biopsy technique and by the level of suspicion of the mammographic lesion.     

Can breast surgeons read mammograms of symptomatic patients in the one-stop breast clinic?

AIM: To establish how accurate surgeons were when compared to the radiologists in interpreting symptomatic mammograms in one-stop clinics. METHODS: The surgeons were asked to write their opinion on the mammograms which was compared with the radiologists' report. 144 patients were involved in the study and the data were analysed by McNemara's test for paired categorical data. RESULTS AND CONCLUSIONS: Surgeons were accurate in interpreting most of the mammographic findings. However, they underestimated the presence of benign calcification which was statistically significant. Surgeons can, therefore, be involved in double reading of mammograms in symptomatic breast disease patients and improve the sensitivity which has been the case in double reading by radiologists in the breast screening programme.