Statistics for quantifying heterogeneity in univariate and bivariate meta‐analyses of binary data: The case of meta‐analyses of diagnostic accuracy

Heterogeneity in diagnostic meta‐analyses is common because of the observational nature of diagnostic studies and the lack of standardization in the positivity criterion (cut‐off value) for some tests. So far the unexplained heterogeneity across studies has been quantified by either using the I² statistic for a single parameter (i.e. either the sensitivity or the specificity) or visually examining the data in a receiver‐operating characteristic space. In this paper, we derive improved I² statistics measuring heterogeneity for dichotomous outcomes, with a focus on diagnostic tests. We show that the currently used estimate of the ‘typical’ within‐study variance proposed by Higgins and Thompson is not able to properly account for the variability of the within‐study variance across studies for dichotomous variables. Therefore, when the between‐study variance is large, the ‘typical’ within‐study variance underestimates the expected within‐study variance, and the corresponding I² is overestimated. We propose to use the expected value of the within‐study variation in the construction of I² in cases of univariate and bivariate diagnostic meta‐analyses. For bivariate diagnostic meta‐analyses, we derive a bivariate version of I² that is able to account for the correlation between sensitivity and specificity. We illustrate the performance of these new estimators using simulated data as well as two real data sets. Copyright © 2014 John Wiley & Sons, Ltd.