Meta‐analysis for diagnostic accuracy studies: a new statistical model using beta‐binomial distributions and bivariate copulas

There are still challenges when meta‐analyzing data from studies on diagnostic accuracy. This is mainly due to the bivariate nature of the response where information on sensitivity and specificity must be summarized while accounting for their correlation within a single trial. In this paper, we propose a new statistical model for the meta‐analysis for diagnostic accuracy studies. This model uses beta‐binomial distributions for the marginal numbers of true positives and true negatives and links these margins by a bivariate copula distribution. The new model comes with all the features of the current standard model, a bivariate logistic regression model with random effects, but has the additional advantages of a closed likelihood function and a larger flexibility for the correlation structure of sensitivity and specificity. In a simulation study, which compares three copula models and two implementations of the standard model, the Plackett and the Gauss copula do rarely perform worse but frequently better than the standard model. We use an example from a meta‐analysis to judge the diagnostic accuracy of telomerase (a urinary tumor marker) for the diagnosis of primary bladder cancer for illustration. Copyright © 2013 John Wiley & Sons, Ltd.     

Meta‐analysis for the comparison of two diagnostic tests—A new approach based on copulas

Meta‐analysis of diagnostic studies is still field of ongoing biometrical research. Especially, clinical researchers call for methods that allow for a comparison of different diagnostic tests to a common gold standard. Focussing on two diagnostic tests, the main parameters of interest are differences of sensitivities and specificities (with their corresponding confidence intervals) between the two diagnostic tests while accounting for the various associations across the two tests and the single studies. Similar to our previous work using generalized linear mixed models to this task, we propose a model with a quadrivariate response consisting of the two sensitivities and the two specificities of both tests. This new approach uses the ideas of copula modelling, and especially a quadrivariate Gaussian copula and a quadrivariate vine copula, which is built from bivariate Plackett copulas. The different copulas are compared in a simulation study and illustrated by the application of population‐based screening for type 2 diabetes.     

Meta‐analysis of diagnostic accuracy studies accounting for disease prevalence: Alternative parameterizations and model selection

In a meta‐analysis of diagnostic accuracy studies, the sensitivities and specificities of a diagnostic test may depend on the disease prevalence since the severity and definition of disease may differ from study to study due to the design and the population considered. In this paper, we extend the bivariate nonlinear random effects model on sensitivities and specificities to jointly model the disease prevalence, sensitivities and specificities using trivariate nonlinear random‐effects models. Furthermore, as an alternative parameterization, we also propose jointly modeling the test prevalence and the predictive values, which reflect the clinical utility of a diagnostic test. These models allow investigators to study the complex relationship among the disease prevalence, sensitivities and specificities; or among test prevalence and the predictive values, which can reveal hidden information about test performance. We illustrate the proposed two approaches by reanalyzing the data from a meta‐analysis of radiological evaluation of lymph node metastases in patients with cervical cancer and a simulation study. The latter illustrates the importance of carefully choosing an appropriate normality assumption for the disease prevalence, sensitivities and specificities, or the test prevalence and the predictive values. In practice, it is recommended to use model selection techniques to identify a best‐fitting model for making statistical inference. In summary, the proposed trivariate random effects models are novel and can be very useful in practice for meta‐analysis of diagnostic accuracy studies. Copyright © 2009 John Wiley & Sons, Ltd.     

Meta‐analysis of studies with bivariate binary outcomes: a marginal beta‐binomial model approach

When conducting a meta‐analysis of studies with bivariate binary outcomes, challenges arise when the within‐study correlation and between‐study heterogeneity should be taken into account. In this paper, we propose a marginal beta‐binomial model for the meta‐analysis of studies with binary outcomes. This model is based on the composite likelihood approach and has several attractive features compared with the existing models such as bivariate generalized linear mixed model (Chu and Cole, 2006) and Sarmanov beta‐binomial model (Chen et al., 2012). The advantages of the proposed marginal model include modeling the probabilities in the original scale, not requiring any transformation of probabilities or any link function, having closed‐form expression of likelihood function, and no constraints on the correlation parameter. More importantly, because the marginal beta‐binomial model is only based on the marginal distributions, it does not suffer from potential misspecification of the joint distribution of bivariate study‐specific probabilities. Such misspecification is difficult to detect and can lead to biased inference using currents methods. We compare the performance of the marginal beta‐binomial model with the bivariate generalized linear mixed model and the Sarmanov beta‐binomial model by simulation studies. Interestingly, the results show that the marginal beta‐binomial model performs better than the Sarmanov beta‐binomial model, whether or not the true model is Sarmanov beta‐binomial, and the marginal beta‐binomial model is more robust than the bivariate generalized linear mixed model under model misspecifications. Two meta‐analyses of diagnostic accuracy studies and a meta‐analysis of case–control studies are conducted for illustration. Copyright © 2015 John Wiley & Sons, Ltd.     

A mixed effect model for bivariate meta‐analysis of diagnostic test accuracy studies using a copula representation of the random effects distribution

Diagnostic test accuracy studies typically report the number of true positives, false positives, true negatives and false negatives. There usually exists a negative association between the number of true positives and true negatives, because studies that adopt less stringent criterion for declaring a test positive invoke higher sensitivities and lower specificities. A generalized linear mixed model (GLMM) is currently recommended to synthesize diagnostic test accuracy studies. We propose a copula mixed model for bivariate meta‐analysis of diagnostic test accuracy studies. Our general model includes the GLMM as a special case and can also operate on the original scale of sensitivity and specificity. Summary receiver operating characteristic curves are deduced for the proposed model through quantile regression techniques and different characterizations of the bivariate random effects distribution. Our general methodology is demonstrated with an extensive simulation study and illustrated by re‐analysing the data of two published meta‐analyses. Our study suggests that there can be an improvement on GLMM in fit to data and makes the argument for moving to copula random effects models. Our modelling framework is implemented in the package CopulaREMADA within the open source statistical environment R . Copyright © 2015 John Wiley & Sons, Ltd.     

Nonparametric meta‐analysis for diagnostic accuracy studies

Summarizing the information of many studies using a meta‐analysis becomes more and more important, also in the field of diagnostic studies. The special challenge in meta‐analysis of diagnostic accuracy studies is that in general sensitivity and specificity are co‐primary endpoints. Across the studies both endpoints are correlated, and this correlation has to be considered in the analysis. The standard approach for such a meta‐analysis is the bivariate logistic random effects model. An alternative approach is to use marginal beta‐binomial distributions for the true positives and the true negatives, linked by copula distributions. In this article, we propose a new, nonparametric approach of analysis, which has greater flexibility with respect to the correlation structure, and always converges. In a simulation study, it becomes apparent that the empirical coverage of all three approaches is in general below the nominal level. Regarding bias, empirical coverage, and mean squared error the nonparametric model is often superior to the standard model, and comparable with the copula model. The three approaches are also applied to two example meta‐analyses. Copyright © 2015 John Wiley & Sons, Ltd.     

A simple and robust method for multivariate meta‐analysis of diagnostic test accuracy

Meta‐analysis of diagnostic test accuracy often involves mixture of case–control and cohort studies. The existing bivariate random‐effects models, which jointly model bivariate accuracy indices (e.g., sensitivity and specificity), do not differentiate cohort studies from case–control studies and thus do not utilize the prevalence information contained in the cohort studies. The recently proposed trivariate generalized linear mixed‐effects models are only applicable to cohort studies, and more importantly, they assume a common correlation structure across studies and trivariate normality on disease prevalence, test sensitivity, and specificity after transformation by some pre‐specified link functions. In practice, very few studies provide justifications of these assumptions, and sometimes these assumptions are violated. In this paper, we evaluate the performance of the commonly used random‐effects model under violations of these assumptions and propose a simple and robust method to fully utilize the information contained in case–control and cohort studies. The proposed method avoids making the aforementioned assumptions and can provide valid joint inferences for any functions of overall summary measures of diagnostic accuracy. Through simulation studies, we find that the proposed method is more robust to model misspecifications than the existing methods. We apply the proposed method to a meta‐analysis of diagnostic test accuracy for the detection of recurrent ovarian carcinoma. Copyright © 2016 John Wiley & Sons, Ltd.     

A multivariate method for meta‐analysis and comparison of diagnostic tests

We present here an extension of the classic bivariate random effects meta‐analysis for the log‐transformed sensitivity and specificity that can be applied for two or more diagnostic tests. The advantage of this method is that a closed‐form expression is derived for the calculation of the within‐studies covariances. The method allows the direct calculation of sensitivity and specificity, as well as, the diagnostic odds ratio, the area under curve and the parameters of the summary receiver operator's characteristic curve, along with the means for a formal comparison of these quantities for different tests. There is no need for individual patient data or the simultaneous evaluation of both diagnostic tests in all studies. The method is simple and fast; it can be extended for several diagnostic tests and can be fitted in nearly all statistical packages. The method was evaluated in simulations and applied in a meta‐analysis for the comparison of anti‐cyclic citrullinated peptide antibody and rheumatoid factor for discriminating patients with rheumatoid arthritis, with encouraging results. Simulations suggest that the method is robust and more powerful compared with the standard bivariate approach that ignores the correlation between tests. Copyright © 2016 John Wiley & Sons, Ltd.     

Estimation of infection prevalence and sensitivity in a stratified two‐stage sampling design employing highly specific diagnostic tests when there is no gold standard

In this work, we describe a two‐stage sampling design to estimate the infection prevalence in a population. In the first stage, an imperfect diagnostic test was performed on a random sample of the population. In the second stage, a different imperfect test was performed in a stratified random sample of the first sample. To estimate infection prevalence, we assumed conditional independence between the diagnostic tests and develop method of moments estimators based on expectations of the proportions of people with positive and negative results on both tests that are functions of the tests' sensitivity, specificity, and the infection prevalence. A closed‐form solution of the estimating equations was obtained assuming a specificity of 100% for both tests. We applied our method to estimate the infection prevalence of visceral leishmaniasis according to two quantitative polymerase chain reaction tests performed on blood samples taken from 4756 patients in northern Ethiopia. The sensitivities of the tests were also estimated, as well as the standard errors of all estimates, using a parametric bootstrap. We also examined the impact of departures from our assumptions of 100% specificity and conditional independence on the estimated prevalence. Copyright © 2015 John Wiley & Sons, Ltd.     

A design‐by‐treatment interaction model for network meta‐analysis with random inconsistency effects

Network meta‐analysis is becoming more popular as a way to analyse multiple treatments simultaneously and, in the right circumstances, rank treatments. A difficulty in practice is the possibility of ‘inconsistency’ or ‘incoherence’, where direct evidence and indirect evidence are not in agreement. Here, we develop a random‐effects implementation of the recently proposed design‐by‐treatment interaction model, using these random effects to model inconsistency and estimate the parameters of primary interest. Our proposal is a generalisation of the model proposed by Lumley and allows trials with three or more arms to be included in the analysis. Our methods also facilitate the ranking of treatments under inconsistency. We derive R and I² statistics to quantify the impact of the between‐study heterogeneity and the inconsistency. We apply our model to two examples. © 2014 The Authors. Statistics in Medicine published by John Wiley & Sons, Ltd.     

Statistics for quantifying heterogeneity in univariate and bivariate meta‐analyses of binary data: The case of meta‐analyses of diagnostic accuracy

Heterogeneity in diagnostic meta‐analyses is common because of the observational nature of diagnostic studies and the lack of standardization in the positivity criterion (cut‐off value) for some tests. So far the unexplained heterogeneity across studies has been quantified by either using the I² statistic for a single parameter (i.e. either the sensitivity or the specificity) or visually examining the data in a receiver‐operating characteristic space. In this paper, we derive improved I² statistics measuring heterogeneity for dichotomous outcomes, with a focus on diagnostic tests. We show that the currently used estimate of the ‘typical’ within‐study variance proposed by Higgins and Thompson is not able to properly account for the variability of the within‐study variance across studies for dichotomous variables. Therefore, when the between‐study variance is large, the ‘typical’ within‐study variance underestimates the expected within‐study variance, and the corresponding I² is overestimated. We propose to use the expected value of the within‐study variation in the construction of I² in cases of univariate and bivariate diagnostic meta‐analyses. For bivariate diagnostic meta‐analyses, we derive a bivariate version of I² that is able to account for the correlation between sensitivity and specificity. We illustrate the performance of these new estimators using simulated data as well as two real data sets. Copyright © 2014 John Wiley & Sons, Ltd.     

Meta‐analysis with missing study‐level sample variance data

We consider a study‐level meta‐analysis with a normally distributed outcome variable and possibly unequal study‐level variances, where the object of inference is the difference in means between a treatment and control group. A common complication in such an analysis is missing sample variances for some studies. A frequently used approach is to impute the weighted (by sample size) mean of the observed variances (mean imputation). Another approach is to include only those studies with variances reported (complete case analysis). Both mean imputation and complete case analysis are only valid under the missing‐completely‐at‐random assumption, and even then the inverse variance weights produced are not necessarily optimal. We propose a multiple imputation method employing gamma meta‐regression to impute the missing sample variances. Our method takes advantage of study‐level covariates that may be used to provide information about the missing data. Through simulation studies, we show that multiple imputation, when the imputation model is correctly specified, is superior to competing methods in terms of confidence interval coverage probability and type I error probability when testing a specified group difference. Finally, we describe a similar approach to handling missing variances in cross‐over studies. Copyright © 2016 John Wiley & Sons, Ltd.     

Meta‐analysis of single‐arm survival studies: a distribution‐free approach for estimating summary survival curves with random effects

In epidemiologic studies and clinical trials with time‐dependent outcome (for instance death or disease progression), survival curves are used to describe the risk of the event over time. In meta‐analyses of studies reporting a survival curve, the most informative finding is a summary survival curve. In this paper, we propose a method to obtain a distribution‐free summary survival curve by expanding the product‐limit estimator of survival for aggregated survival data. The extension of DerSimonian and Laird's methodology for multiple outcomes is applied to account for the between‐study heterogeneity. Statistics I² and H² are used to quantify the impact of the heterogeneity in the published survival curves. A statistical test for between‐strata comparison is proposed, with the aim to explore study‐level factors potentially associated with survival. The performance of the proposed approach is evaluated in a simulation study. Our approach is also applied to synthesize the survival of untreated patients with hepatocellular carcinoma from aggregate data of 27 studies and synthesize the graft survival of kidney transplant recipients from individual data from six hospitals. Copyright © 2014 John Wiley & Sons, Ltd.     

An Empirical Comparison of Meta‐analysis and Mega‐analysis of Individual Participant Data for Identifying Gene‐Environment Interactions

For analysis of the main effects of SNPs, meta‐analysis of summary results from individual studies has been shown to provide comparable results as “mega‐analysis” that jointly analyzes the pooled participant data from the available studies. This fact revolutionized the genetic analysis of complex traits through large GWAS consortia. Investigations of gene‐environment (G×E) interactions are on the rise since they can potentially explain a part of the missing heritability and identify individuals at high risk for disease. However, for analysis of gene‐environment interactions, it is not known whether these methods yield comparable results. In this empirical study, we report that the results from both methods were largely consistent for all four tests; the standard 1 degree of freedom (df) test of main effect only, the 1 df test of the main effect (in the presence of interaction effect), the 1 df test of the interaction effect, and the joint 2 df test of main and interaction effects. They provided similar effect size and standard error estimates, leading to comparable P‐values. The genomic inflation factors and the number of SNPs with various thresholds were also comparable between the two approaches. Mega‐analysis is not always feasible especially in very large and diverse consortia since pooling of raw data may be limited by the terms of the informed consent. Our study illustrates that meta‐analysis can be an effective approach also for identifying interactions. To our knowledge, this is the first report investigating meta‐versus mega‐analyses for interactions.     

Beta‐binomial model for meta‐analysis of odds ratios

In meta‐analysis of odds ratios (ORs), heterogeneity between the studies is usually modelled via the additive random effects model (REM). An alternative, multiplicative REM for ORs uses overdispersion. The multiplicative factor in this overdispersion model (ODM) can be interpreted as an intra‐class correlation (ICC) parameter. This model naturally arises when the probabilities of an event in one or both arms of a comparative study are themselves beta‐distributed, resulting in beta‐binomial distributions. We propose two new estimators of the ICC for meta‐analysis in this setting. One is based on the inverted Breslow‐Day test, and the other on the improved gamma approximation by Kulinskaya and Dollinger (2015, p. 26) to the distribution of Cochran's Q. The performance of these and several other estimators of ICC on bias and coverage is studied by simulation. Additionally, the Mantel‐Haenszel approach to estimation of ORs is extended to the beta‐binomial model, and we study performance of various ICC estimators when used in the Mantel‐Haenszel or the inverse‐variance method to combine ORs in meta‐analysis. The results of the simulations show that the improved gamma‐based estimator of ICC is superior for small sample sizes, and the Breslow‐Day‐based estimator is the best for . The Mantel‐Haenszel‐based estimator of OR is very biased and is not recommended. The inverse‐variance approach is also somewhat biased for ORs≠1, but this bias is not very large in practical settings. Developed methods and R programs, provided in the Web Appendix, make the beta‐binomial model a feasible alternative to the standard REM for meta‐analysis of ORs. © 2017 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.     

Random effects meta‐analysis of event outcome in the framework of the generalized linear mixed model with applications in sparse data

We consider random effects meta‐analysis where the outcome variable is the occurrence of some event of interest. The data structures handled are where one has one or more groups in each study, and in each group either the number of subjects with and without the event, or the number of events and the total duration of follow‐up is available. Traditionally, the meta‐analysis follows the summary measures approach based on the estimates of the outcome measure(s) and the corresponding standard error(s). This approach assumes an approximate normal within‐study likelihood and treats the standard errors as known. This approach has several potential disadvantages, such as not accounting for the standard errors being estimated, not accounting for correlation between the estimate and the standard error, the use of an (arbitrary) continuity correction in case of zero events, and the normal approximation being bad in studies with few events. We show that these problems can be overcome in most cases occurring in practice by replacing the approximate normal within‐study likelihood by the appropriate exact likelihood. This leads to a generalized linear mixed model that can be fitted in standard statistical software. For instance, in the case of odds ratio meta‐analysis, one can use the non‐central hypergeometric distribution likelihood leading to mixed‐effects conditional logistic regression. For incidence rate ratio meta‐analysis, it leads to random effects logistic regression with an offset variable. We also present bivariate and multivariate extensions. We present a number of examples, especially with rare events, among which an example of network meta‐analysis. Copyright © 2010 John Wiley & Sons, Ltd.     

Meta‐analysis of studies of a specific delivery mode for a modified‐carbohydrate diet

Background: Obesity is highly prevalent throughout the world. Although modified‐carbohydrate diets (MCDs) comprise one popular approach, questions remain about their utility for weight loss. The objective of the present study was to conduct a meta‐analysis of randomised controlled trials (RCTs) of a specific MCD compared with various control diets on weight loss. Methods: Data from four RCTs (three obtained from the sponsor and one indentified through literature searches) were included. Intent‐to‐treat analyses were conducted using multiple imputation to handle missing data, where possible. Because inter‐study heterogeneity was demonstrated with fixed‐effects meta‐analysis, a random‐effects meta‐analysis also was conducted. Results: When considered separately, all four studies showed greater reduction in body weight with the MCD compared to control diets at 12‐week follow‐up; the results at 24 weeks (available for three of the studies) were not as consistent. Results for body mass index (BMI) were similar. Greater reductions in waist circumference with the MCD were seen at either time point in only one study. When fixed‐effects meta‐analysis was applied, significantly greater reductions in weight, BMI and waist circumference with the MCD at both 12 weeks (1.66 kg, 0.53 kg m^–2 and 1.02 cm, respectively) and 24 weeks (1.20 kg, 0.43 kg m^–2 and 0.69 cm, respectively) were evident. Random‐effects meta‐analysis revealed similar results; however, the 24‐week difference for a reduction in waist circumference was no longer statistically significant. Conclusions: Meta‐analysis of individual RCT results demonstrated consistent benefits of this MCD compared to control diets on weight loss up to 24 weeks and waist circumference up to 12 weeks.     

Meta‐STEPP: subpopulation treatment effect pattern plot for individual patient data meta‐analysis

We have developed a method, called Meta‐STEPP (subpopulation treatment effect pattern plot for meta‐analysis), to explore treatment effect heterogeneity across covariate values in the meta‐analysis setting for time‐to‐event data when the covariate of interest is continuous. Meta‐STEPP forms overlapping subpopulations from individual patient data containing similar numbers of events with increasing covariate values, estimates subpopulation treatment effects using standard fixed‐effects meta‐analysis methodology, displays the estimated subpopulation treatment effect as a function of the covariate values, and provides a statistical test to detect possibly complex treatment‐covariate interactions. Simulation studies show that this test has adequate type‐I error rate recovery as well as power when reasonable window sizes are chosen. When applied to eight breast cancer trials, Meta‐STEPP suggests that chemotherapy is less effective for tumors with high estrogen receptor expression compared with those with low expression. Copyright © 2016 John Wiley & Sons, Ltd.     

Sensitivity analysis for publication bias in meta‐analysis of diagnostic studies for a continuous biomarker

Publication bias is one of the most important issues in meta‐analysis. For standard meta‐analyses to examine intervention effects, the funnel plot and the trim‐and‐fill method are simple and widely used techniques for assessing and adjusting for the influence of publication bias, respectively. However, their use may be subjective and can then produce misleading insights. To make a more objective inference for publication bias, various sensitivity analysis methods have been proposed, including the Copas selection model. For meta‐analysis of diagnostic studies evaluating a continuous biomarker, the summary receiver operating characteristic (sROC) curve is a very useful method in the presence of heterogeneous cutoff values. To our best knowledge, no methods are available for evaluation of influence of publication bias on estimation of the sROC curve. In this paper, we introduce a Copas‐type selection model for meta‐analysis of diagnostic studies and propose a sensitivity analysis method for publication bias. Our method enables us to assess the influence of publication bias on the estimation of the sROC curve and then judge whether the result of the meta‐analysis is sufficiently confident or should be interpreted with much caution. We illustrate our proposed method with real data.     

Meta‐analysis of aggregate data on medical events

Meta‐analyses of clinical trials often treat the number of patients experiencing a medical event as binomially distributed when individual patient data for fitting standard time‐to‐event models are unavailable. Assuming identical drop‐out time distributions across arms, random censorship, and low proportions of patients with an event, a binomial approach results in a valid test of the null hypothesis of no treatment effect with minimal loss in efficiency compared with time‐to‐event methods. To deal with differences in follow‐up—at the cost of assuming specific distributions for event and drop‐out times—we propose a hierarchical multivariate meta‐analysis model using the aggregate data likelihood based on the number of cases, fatal cases, and discontinuations in each group, as well as the planned trial duration and groups sizes. Such a model also enables exchangeability assumptions about parameters of survival distributions, for which they are more appropriate than for the expected proportion of patients with an event across trials of substantially different length. Borrowing information from other trials within a meta‐analysis or from historical data is particularly useful for rare events data. Prior information or exchangeability assumptions also avoid the parameter identifiability problems that arise when using more flexible event and drop‐out time distributions than the exponential one. We discuss the derivation of robust historical priors and illustrate the discussed methods using an example. We also compare the proposed approach against other aggregate data meta‐analysis methods in a simulation study. Copyright © 2016 John Wiley & Sons, Ltd.