Inter- and intraradiologist variability in the BI-RADS assessment and breast density categories for screening mammograms

Objective

The aim of this study was to evaluate reader variability in screening mammograms according to the American College of Radiology Breast Imaging Reporting and Data System (BI-RADS) assessment and breast density categories.

Methods

A stratified random sample of 100 mammograms was selected from a population-based breast cancer screening programme in Barcelona, Spain: 13 histopathologically confirmed breast cancers and 51 with true-negative and 36 with false-positive results. 21 expert radiologists from radiological units of breast cancer screening programmes in Catalonia, Spain, reviewed the mammography images twice within a 6-month interval. The readers described each mammography using BI-RADS assessment and breast density categories. Inter- and intraradiologist agreement was assessed using percentage of concordance and the kappa (κ) statistic.

Results

Fair interobserver agreement was observed for the BI-RADS assessment [κ=0.37, 95% confidence interval (CI) 0.36–0.38]. When the categories were collapsed in terms of whether additional evaluation was required (Categories III, 0, IV, V) or not (I and II), moderate agreement was found (κ=0.53, 95% CI 0.52–0.54). Intra-observer agreement for BI-RADS assessment was moderate using all categories (κ=0.53, 95% CI 0.50–0.55) and substantial on recall (κ=0.66, 95% CI 0.63–0.70). Regarding breast density, inter- and intraradiologist agreement was substantial (κ=0.73, 95% CI 0.72–0.74 and κ=0.69, 95% CI 0.68–0.70, respectively).

Conclusion

We observed a substantial intra-observer agreement in the BI-RADS assessment but only moderate interobserver agreement. Both inter- and intra-observer agreement in mammographic interpretation of breast density was substantial.

Advances in knowledge

Educational efforts should be made to decrease radiologists'' variability in BI-RADS assessment interpretation in population-based breast screening programmes.Breast cancer is the most commonly diagnosed cancer and the leading cause of cancer death in females worldwide, accounting for 23% of the total new cancer cases and 14% of the total cancer-related deaths in 2008 [1]. In Spain, more than 20 000 cases are diagnosed and approximately 6000 females die each year because of this tumour [2].Breast cancer screening by mammography is the only evidence-based screening procedure currently available to reduce breast cancer mortality. Accuracy of screening mammography depends on various factors, such as the protocols for mammogram reading, the characteristics of the female and of the breast, and the experience of radiologists [3-5]. Great efforts have been made to improve its accuracy. One of these is the implementation of double reading as it increases the cancer detection rate and reduces the further assessment rate [6]. Furthermore, the American College of Radiology developed the Breast Imaging Reporting and Data System (BI-RADS) in order to reduce discordance in the interpretation of mammographic findings, to standardise mammographic reporting and to facilitate follow-up [7,8].A limited number of studies have analysed observer variability in mammography interpretation using BI-RADS assessment as well as breast density categories [9-13]. Thus, the aim of this study was to assess the inter- and intra-observer agreement regarding the assessment and breast density in a breast cancer screening programme in the city of Barcelona, Spain. Furthermore, we wanted to investigate the association between female characteristics and BI-RADS discordance.