Coronary heart disease in primary care: accuracy of medical history and physical findings in patients with chest pain - a study protocol for a systematic review with individual patient data

ABSTRACT: BACKGROUND: Chest pain is a common complaint in primary care, with coronary heart disease (CHD) being the most concerning of many potential causes. Systematic reviews on the sensitivity and specificity of symptoms and signs summarize the evidence about which of them are most useful in making a diagnosis. Previous meta-analyses are dominated by studies of patients referred to specialists. Moreover, as the analysis is typically based on study-level data, the statistical analyses in these reviews are limited while meta-analyses based on individual patient data can provide additional information. Our patient-level meta-analysis has three unique aims. First, we strive to determine the diagnostic accuracy of symptoms and signs for myocardial ischemia in primary care. Second, we investigate associations between study- or patient-level characteristics and measures of diagnostic accuracy. Third, we aim to validate existing clinical prediction rules for diagnosing myocardial ischemia in primary care. This article describes the methods of our study and six prospective studies of primary care patients with chest pain. Later articles will describe the main results. METHODS: We will conduct a systematic review and IPD meta-analysis of studies evaluating the diagnostic accuracy of symptoms and signs for diagnosing coronary heart disease in primary care. We will perform bivariate analyses to determine the sensitivity, specificity and likelihood ratios of individual symptoms and signs and multivariate analyses to explore the diagnostic value of an optimal combination of all symptoms and signs based on all data of all studies. We will validate existing clinical prediction rules from each of the included studies by calculating measures of diagnostic accuracy separately by study. DISCUSSION: Our study will face several methodological challenges. First, the number of studies will be limited. Second, the investigators of original studies defined some outcomes and predictors differently. Third, the studies did not collect the same standard clinical data set. Fourth, missing data, varying from partly missing to fully missing, will have to be dealt with. Despite these limitations, we aim to summarize the available evidence regarding the diagnostic accuracy of symptoms and signs for diagnosing CHD in patients presenting with chest pain in primary care. Review registration Centre for Reviews and Dissemination (University of York): CRD42011001170.     

SOME ISSUES IN ESTIMATING THE EFFECT OF PROGNOSTIC FACTORS FROM INCOMPLETE COVARIATE DATA

In evaluating prognostic factors by means of regression models, missing values in the covariate data are a frequent complication. There exist statistical tools to analyse such incomplete data in an efficient manner, and in this paper we make use of the traditional maximum likelihood principle. As well as an analysis including the incompletely measured covariates, such tools also allow further strategies of data analysis. For example, we can use surrogate variables to improve the prediction of missing values or we can try to investigate a questionable ‘missing at random’ assumption. We discuss these techniques using the example of a clinical study where one important covariate is missing for about half the subjects. Additionally we consider two further issues: evaluation of differences between estimates from a complete case analysis and analyses using all subjects and assessment of the predictive value of missing values. © 1997 by John Wiley & Sons, Ltd.     

How can I deal with missing data in my study?

Missing data in medical research is a common problem that has long been recognised by statisticians and medical researchers alike. In general, if the effect of missing data is not taken into account the results of the statistical analyses will be biased and the amount of variability in the data will not be correctly estimated. There are three main types of missing data pattern: Missing Completely At Random (MCAR), Missing At Random (MAR) and Not Missing At Random (NMAR). The type of missing data that a researcher has in their dataset determines the appropriate method to use in handling the missing data before a formal statistical analysis begins. The aim of this practice note is to describe these patterns of missing data and how they can occur, as well describing the methods of handling them. Simple and more complex methods are described, including the advantages and disadvantages of each method as well as their availability in routine software. It is good practice to perform a sensitivity analysis employing different missing data techniques in order to assess the robustness of the conclusions drawn from each approach.     

Imputation of missing values is superior to complete case analysis and the missing-indicator method in multivariable diagnostic research: a clinical example

BACKGROUND AND OBJECTIVES: To illustrate the effects of different methods for handling missing data--complete case analysis, missing-indicator method, single imputation of unconditional and conditional mean, and multiple imputation (MI)--in the context of multivariable diagnostic research aiming to identify potential predictors (test results) that independently contribute to the prediction of disease presence or absence. METHODS: We used data from 398 subjects from a prospective study on the diagnosis of pulmonary embolism. Various diagnostic predictors or tests had (varying percentages of) missing values. Per method of handling these missing values, we fitted a diagnostic prediction model using multivariable logistic regression analysis. RESULTS: The receiver operating characteristic curve area for all diagnostic models was above 0.75. The predictors in the final models based on the complete case analysis, and after using the missing-indicator method, were very different compared to the other models. The models based on MI did not differ much from the models derived after using single conditional and unconditional mean imputation. CONCLUSION: In multivariable diagnostic research complete case analysis and the use of the missing-indicator method should be avoided, even when data are missing completely at random. MI methods are known to be superior to single imputation methods. For our example study, the single imputation methods performed equally well, but this was most likely because of the low overall number of missing values.     

Multiple imputation techniques in small sample clinical trials

Clinical trials allow researchers to draw conclusions about the effectiveness of a treatment. However, the statistical analysis used to draw these conclusions will inevitably be complicated by the common problem of attrition. Resorting to ad hoc methods such as case deletion or mean imputation can lead to biased results, especially if the amount of missing data is high. Multiple imputation, on the other hand, provides the researcher with an approximate solution that can be generalized to a number of different data sets and statistical problems. Multiple imputation is known to be statistically valid when n is large. However, questions still remain about the validity of multiple imputation for small samples in clinical trials. In this paper we investigate the small-sample performance of several multiple imputation methods, as well as the last observation carried forward method.     

Roaming through methodology. XVI. What to do about missing data]

Medical scientific research involving multiple measurements in patients is usually complicated by missing values. In case of missing values the choice is to limit the analysis to the complete cases or to analyse all available data. Both methods may suffer from substantial bias and may only be applied in a valid way if the rather strong assumption of 'missing completely at random' holds for the missing values, i.e. the missing value is not related to the other measured data nor to unmeasured data. Two other statistical methods may be applied to deal with missing values: the likelihood approach and the multiple imputation method. These methods make efficient use of all available data and take into account information implied by the available data. These methods are valid under the less stringent assumption of 'missing at random', i.e. the missing value is related to the other measured data, but not to unmeasured data. The best approach is to ensure that no data are missing.     

Dealing with missing outcome data in randomized trials and observational studies

Although missing outcome data are an important problem in randomized trials and observational studies, methods to address this issue can be difficult to apply. Using simulated data, the authors compared 3 methods to handle missing outcome data: 1) complete case analysis; 2) single imputation; and 3) multiple imputation (all 3 with and without covariate adjustment). Simulated scenarios focused on continuous or dichotomous missing outcome data from randomized trials or observational studies. When outcomes were missing at random, single and multiple imputations yielded unbiased estimates after covariate adjustment. Estimates obtained by complete case analysis with covariate adjustment were unbiased as well, with coverage close to 95%. When outcome data were missing not at random, all methods gave biased estimates, but handling missing outcome data by means of 1 of the 3 methods reduced bias compared with a complete case analysis without covariate adjustment. Complete case analysis with covariate adjustment and multiple imputation yield similar estimates in the event of missing outcome data, as long as the same predictors of missingness are included. Hence, complete case analysis with covariate adjustment can and should be used as the analysis of choice more often. Multiple imputation, in addition, can accommodate the missing-not-at-random scenario more flexibly, making it especially suited for sensitivity analyses.     

Missing predictors in models of effect size

Missing data occur frequently in meta-analysis. Reviewers inevitably face decisions about how to handle missing data, especially when predictors in a model of effect size are missing from some of the identified studies. Commonly used methods for missing data such as complete case analysis and mean substitution often yield biased estimates. This article briefly reviews the particular problems missing predictors cause in a meta-analysis, discusses the properties of commonly used missing data methods, and provides suggestions for ways to handle missing predictors when estimating effect size models. Maximum likelihood methods for multivariate normal data and multiple imputation hold the most promise for handling missing predictors in meta-analysis. These two model-based methods apply to a broad set of data situations, are based on sound statistical theory, and utilize all information available to obtain efficient estimators.     

The value of capture-recapture methods even for apparent exhaustive surveys. The need for adjustment for source of ascertainment intersection in attempted complete prevalence studies.

Almost all reported prevalence studies of which we are aware make exhaustive attempts to find diagnosed individuals and report all affected individuals, but make no attempt to estimate or adjust for missing cases. Yet very simple methods introduced in the planning stage of a prevalence study may enable investigators, or at least those subsequently reading their reports, to derive such adjusted estimates. If investigators keep track of the nature of the ascertainment of cases by source and collect and report data that allow calculation of the number of cases by source intersection, then they, or at least others, may derive estimates of missing cases and of the total population affected, by using readily available analogues of capture-recapture methods developed for wildlife populations censuses. Unfortunately, such methods are often inappropriately disparaged or ignored by epidemiologists. The derived estimates are sensitive to assumptions about dependence or independence ("interaction") of various sources, assumptions that sometimes are unprovable, and these estimates have some uncertainty because of statistical fluctuation. Moreover, most investigators who attempt exhaustive prevalence studies apparently believe that they have ascertained all cases and that there is no need to attempt to adjust for, let alone provide data pertinent to, the number of missing cases or to use a statistical method that will at best imply a certain imprecision to their result. Yet a survey that reports prevalence data without adjustment for, or data on, source intersection in essence makes an estimate of missing cases--zero--while providing no quantitative grounds for that claim. The results of all such surveys should be regarded with skepticism because, at best (if the case reports are accurate), they provide only a lower boundary of prevalence. We illustrate the grounds for these views by analyzing data from an apparently exhaustive prevalence study that used at least 14 distinct sources for ascertainment, including advertising, to find cases. Available limited data on source intersection provided in the report enable the plausible inference that the study missed about 25-40% of cases. We urge that no attempted complete prevalence studies be presented without data on ascertainment by source intersection.     

A bias‐corrected estimator in multiple imputation for missing data

Multiple imputation (MI) is one of the most popular methods to deal with missing data, and its use has been rapidly increasing in medical studies. Although MI is rather appealing in practice since it is possible to use ordinary statistical methods for a complete data set once the missing values are fully imputed, the method of imputation is still problematic. If the missing values are imputed from some parametric model, the validity of imputation is not necessarily ensured, and the final estimate for a parameter of interest can be biased unless the parametric model is correctly specified. Nonparametric methods have been also proposed for MI, but it is not so straightforward as to produce imputation values from nonparametrically estimated distributions. In this paper, we propose a new method for MI to obtain a consistent (or asymptotically unbiased) final estimate even if the imputation model is misspecified. The key idea is to use an imputation model from which the imputation values are easily produced and to make a proper correction in the likelihood function after the imputation by using the density ratio between the imputation model and the true conditional density function for the missing variable as a weight. Although the conditional density must be nonparametrically estimated, it is not used for the imputation. The performance of our method is evaluated by both theory and simulation studies. A real data analysis is also conducted to illustrate our method by using the Duke Cardiac Catheterization Coronary Artery Disease Diagnostic Dataset.     

Examining the Impact of Missing Data on Propensity Score Estimation in Determining the Effectiveness of Self-Monitoring of Blood Glucose (SMBG)

In many health services applications, research to determine the effectiveness of a particular treatment cannot be carried out using a controlled clinical trial. In settings such as these, observational studies must be used. Propensity score methods are useful tools to employ in order to balance the distribution of covariates between treatment groups and hence reduce the potential bias in treatment effect estimates in observational studies. A challenge in many health services research studies is the presence of missing data among the covariates that need to be balanced. In this paper, we compare three simple propensity models using data that examine the effectiveness of self-monitoring of blood glucose (SMBG) in reducing hemoglobin A1c in a cohort of 10,566 type 2 diabetics. The first propensity score model uses only subjects with complete case data (n=6,687), the second incorporates missing value indicators into the model, and the third fits separate propensity scores for each pattern of missing data. We compare the results of these methods and find that incorporating missing data into the propensity score model reduces the estimated effect of SMBG on hemoglobin A1c by more than 10%, although this reduction was not clinically significant. In addition, beginning with the complete data, we artificially introduce missing data using a nonignorable missing data mechanism and compare treatment effect estimates using the three propensity score methods and a simple analysis of covariance (ANCOVA) method. In these analyses, we find that the complete case analysis and the ANCOVA method both perform poorly, the missing value indicator model performs moderately well, and the pattern mixture model performs even better in estimating the original treatment effect observed in thecomplete data prior to the introduction of artificial missing data. We conclude that in observational studies onemust not only adjust for potentially confounding variables using methods such as propensity scores, but oneshould also account for missing data in these models in order to allow for causal inference more appropriately to be applied.     

THE ANALYSIS OF INCOMPLETE DATA IN THE THREE-PERIOD TWO-TREATMENT CROSS-OVER DESIGN FOR CLINICAL TRIALS

The additional time to complete a three-period two-treatment (3P2T) cross-over trial may cause a greater number of patient dropouts than with a two-period trial. This paper develops maximum likelihood (ML), single imputation and multiple imputation missing data analysis methods for the 3P2T cross-over designs. We use a simulation study to compare and contrast these methods with one another and with the benchmark method of missing data analysis for cross-over trials, the complete case (CC) method. Data patterns examined include those where the missingness differs between the drug types and depends on the unobserved data. Depending on the missing data mechanism and the rate of missingness of the data, one can realize substantial improvements in information recovery by using data from the partially completed patients. We recommend these approaches for the 3P2T cross-over designs.     

Estimating vaccine efficacy using auxiliary outcome data and a small validation sample

In vaccine studies, a specific diagnosis of a suspected case by culture or serology of the infectious agent is expensive and difficult. Implementing validation sets in the study is less expensive and is easier to carry out. In studies using validation sets, the non-specific or auxiliary outcome is measured on each participant while the specific outcome is measured only for a small proportion of the participants. Vaccine efficacy, defined as one minus some measure of relative risk, could be severely attenuated if based only on the auxiliary outcome. Applying missing data analysis techniques could thus correct the bias while maintaining statistical efficiency. However, when the sample size in the validation sets is small and the vaccine is highly efficacious, all specific outcomes are likely to be negative in the validation set in the vaccinated group. Two commonly used missing data analysis methods, the mean score method and multiple imputation, depend on the ad hoc continuity correction when none of the specific outcomes are positive and the normality or log-normality assumption of relative risk, which may not hold when the relative risk is highly skewed, to estimate the confidence interval. In this paper, we propose a Bayesian method to estimate vaccine efficacy and its highest probability density (HPD) credible set using Monte Carlo (MC) methods when using auxiliary outcome data and a small validation sample. Comparing the performance of these approaches using data from a field study of influenza vaccine and simulations, we recommend to use the Bayesian method in this situation.     

Meta‐regression with partial information on summary trial or patient characteristics

We present a model for meta‐regression in the presence of missing information on some of the study level covariates, obtaining inferences using Bayesian methods. In practice, when confronted with missing covariate data in a meta‐regression, it is common to carry out a complete case or available case analysis. We propose to use the full observed data, modelling the joint density as a factorization of a meta‐regression model and a conditional factorization of the density for the covariates. With the inclusion of several covariates, inter‐relations between these covariates are modelled. Under this joint likelihood‐based approach, it is shown that the lesser assumption of the covariates being Missing At Random is imposed, instead of the more usual Missing Completely At Random (MCAR) assumption. The model is easily programmable in WinBUGS, and we examine, through the analysis of two real data sets, sensitivity and robustness of results to the MCAR assumption. Copyright © 2010 John Wiley & Sons, Ltd.     

The impact of missing data in the estimation of concentration index: a potential source of bias

The purpose of this paper is to raise awareness of missing data when concentration indices are used to evaluate health-related inequality. Concentration indices are most commonly calculated using individual-level survey data. Incomplete data is a pervasive problem faced by most applied researchers who use survey data. The default analysis method in most statistical software packages is complete-case analysis. This excludes any cases where any variables are missing. If the missing variables in question are not completely random, the calculated concentration indices are likely to be biased, which may lead to inappropriate policy recommendations. In this paper, I use both a case study and a simulation study to show how complete-case analysis may lead to biases in the estimation of concentration indices. A possible solution to correct such biases is proposed.     

Joint Longitudinal Models for Dealing With Missing at Random Data in Trial-Based Economic Evaluations

ObjectivesIn trial-based economic evaluation, some individuals are typically associated with missing data at some time point, so that their corresponding aggregated outcomes (eg, quality-adjusted life-years) cannot be evaluated. Restricting the analysis to the complete cases is inefficient and can result in biased estimates, while imputation methods are often implemented under a missing at random (MAR) assumption. We propose the use of joint longitudinal models to extend standard approaches by taking into account the longitudinal structure to improve the estimation of the targeted quantities under MAR.MethodsWe compare the results from methods that handle missingness at an aggregated (case deletion, baseline imputation, and joint aggregated models) and disaggregated (joint longitudinal models) level under MAR. The methods are compared using a simulation study and applied to data from 2 real case studies.ResultsSimulations show that, according to which data affect the missingness process, aggregated methods may lead to biased results, while joint longitudinal models lead to valid inferences under MAR. The analysis of the 2 case studies support these results as both parameter estimates and cost-effectiveness results vary based on the amount of data incorporated into the model.ConclusionsOur analyses suggest that methods implemented at the aggregated level are potentially biased under MAR as they ignore the information from the partially observed follow-up data. This limitation can be overcome by extending the analysis to a longitudinal framework using joint models, which can incorporate all the available evidence.     

The use of missingness screens in clinical epidemiologic research has implications for regression modeling

OBJECTIVE: Properly handling missing data is a challenge, especially when working with older populations that have high levels of morbidity and mortality. We illustrate methods for understanding whether missing values are ignorable and describe implications of their use in regression modeling. STUDY DESIGN AND SETTING: The use of missingness screens such as Little's missing completely at random "MCAR test" (1988) and the "Index of Sensitivity to Nonignorability (ISNI)" by Troxel and colleagues (2004)introduces complications for regression modeling, and, particularly, for risk factor selection. In a case study of older patients with simulated missing values for a delirium outcome set in a 14-bed medical intensive care unit, we outline a model fitting process that incorporates the use of missingness screens, controls for collinearity, and selects variables based on model fit. RESULTS: The proposed model fitting process identifies more actual risk factors for ICU delirium than does a complete case analysis. CONCLUSION: Use of imputation and other methods for handling missing data assist in the identification of risk factors. They do so accurately only when correct assumptions are made about the nature of missing data. Missingness screens enable researchers to investigate these assumptions.     

Analysis of longitudinal Gaussian data with missing data on the response variable

BACKGROUND: Using an application and a simulation study we show the bias induced by missing data in the outcome in longitudinal studies and discuss suitable statistical methods according to the type of missing responses when the variable under study is gaussian. Method: The model used for the analysis of gaussian longitudinal data is the mixed effects linear model. When the probability of response does not depend on the missing values of the outcome and on the parameters of the linear model, missing data are ignorable, and parameters of the mixed effects linear model may be estimated by the maximum likelihood method with classical softwares. When the missing data are non ignorable, several methods have been proposed. We describe the method proposed by Diggle and Kenward (1994) (DK method) for which a software is available. This model consists in the combination of a linear mixed effects model for the outcome variable and a logistic model for the probability of response which depends on the outcome variable. RESULTS: A simulation study shows the efficacy of this method and its limits when the data are not normal. In this case, estimators obtained by the DK approach may be more biased than estimators obtained under the hypothesis of ignorable missing data even if the data are non ignorable. Data of the Paquid cohort about the evolution of the scores to a neuropsychological test among elderly subjects show the bias of a naive analysis using all available data. Although missing responses are not ignorable in this study, estimates of the linear mixed effects model are not very different using the DK approach and the hypothesis of ignorable missing data. CONCLUSION: Statistical methods for longitudinal data including non ignorable missing responses are sensitive to hypotheses difficult to verify. Thus, it will be better in practical applications to perform an analysis under the hypothesis of ignorable missing responses and compare the results obtained with several approaches for non ignorable missing data. However, such a strategy requires development of new softwares.     

L’imputation multiple des données manquantes aléatoirement : concepts généraux et présentation d’une méthode Monte-Carlo

BackgroundStatistical analysis of a data set with missing data is a frequent problem to deal with in epidemiology. Methods are available to manage incomplete observations, avoiding biased estimates and improving their precision, compared to more traditional methods, such as the analysis of the sub-sample of complete observations.MethodsOne of these approaches is multiple imputation, which consists in imputing successively several values for each missing data item. Several completed data sets having the same distribution characteristics as the observed data (variability and correlations) are thus generated. Standard analyses are done separately on each completed dataset then combined to obtain a global result. In this paper, we discuss the various assumptions made on the origin of missing data (at random or not), and we present in a pragmatic way the process of multiple imputation. A recent method, Multiple Imputation by Chained Equations (MICE), based on a Monte-Carlo Markov Chain algorithm under missing at random data (MAR) hypothesis, is described. An illustrative example of the MICE method is detailed for the analysis of the relation between a dichotomous variable and two covariates presenting MAR data with no particular structure, through multivariate logistic regression.ResultsCompared with the original dataset without missing data, the results show a substantial improvement of the regression coefficient estimates with the MICE method, relatively to those obtained on the dataset with complete observations.ConclusionThis method does not require any direct assumption on joint distribution of the variables and it is presently implemented in standard statistical software (Splus, Stata). It can be used for multiple imputation of missing data of several variables with no particular structure.