Performance of weighted estimating equations for longitudinal binary data with drop-outs missing at random

The generalized estimating equations (GEE) approach is commonly used to model incomplete longitudinal binary data. When drop-outs are missing at random through dependence on observed responses (MAR), GEE may give biased parameter estimates in the model for the marginal means. A weighted estimating equations approach gives consistent estimation under MAR when the drop-out mechanism is correctly specified. In this approach, observations or person-visits are weighted inversely proportional to their probability of being observed. Using a simulation study, we compare the performance of unweighted and weighted GEE in models for time-specific means of a repeated binary response with MAR drop-outs. Weighted GEE resulted in smaller finite sample bias than GEE. However, when the drop-out model was misspecified, weighted GEE sometimes performed worse than GEE. Weighted GEE with observation-level weights gave more efficient estimates than a weighted GEE procedure with cluster-level weights.     

Impact of missing data due to drop-outs on estimators for rates of change in longitudinal studies: a simulation study.

Many cohort studies and clinical trials are designed to compare rates of change over time in one or more disease markers in several groups. One major problem in such longitudinal studies is missing data due to patient drop-out. The bias and efficiency of six different methods to estimate rates of changes in longitudinal studies with incomplete observations were compared: generalized estimating equation estimates (GEE) proposed by Liang and Zeger (1986); unweighted average of ordinary least squares (OLSE) of individual rates of change (UWLS); weighted average of OLSE (WLS); conditional linear model estimates (CLE), a covariate type estimates proposed by Wu and Bailey (1989); random effect (RE), and joint multivariate RE (JMRE) estimates. The latter method combines a linear RE model for the underlying pattern of the marker with a log-normal survival model for informative drop-out process. The performance of these methods in the presence of missing data completely at random (MCAR), at random (MAR) and non-ignorable (NIM) were compared in simulation studies. Data for the disease marker were generated under the linear random effects model with parameter values derived from realistic examples in HIV infection. Rates of drop-out, assumed to increase over time, were allowed to be independent of marker values or to depend either only on previous marker values or on both previous and current marker values. Under MACR all six methods yielded unbiased estimates of both group mean rates and between-group difference. However, the cross-sectional view of the data in the GEE method resulted in seriously biased estimates under MAR and NIM drop-out process. The bias in the estimates ranged from 30 per cent to 50 per cent. The degree of bias in the GEE estimates increases with the severity of non-randomness and with the proportion of MAR data. Under MCAR and MAR all the other five methods performed relatively well. RE and JMRE estimates were more efficient(that is, had smaller variance) than UWLS, WLS and CL estimates. Under NIM, WLS and particularly RE estimates tended to underestimate the average rate of marker change (bias approximately 10 per cent). Under NIM, UWLS, CL and JMRE performed better in terms of bias (3-5 per cent) with the JMRE giving the most efficient estimates. Given that markers are key variables related to disease progression, missing marker data are likely to be at least MAR. Thus, the GEE method may not be appropriate for analysing such longitudinal marker data. The potential biases due to incomplete data require greater recognition in reports of longitudinal studies. Sensitivity analyses to assess the effect of drop-outs on inferences about the target parameters are important.     

Multiple imputation under Bayesianly smoothed pattern-mixture models for non-ignorable drop-out

Conventional pattern-mixture models can be highly sensitive to model misspecification. In many longitudinal studies, where the nature of the drop-out and the form of the population model are unknown, interval estimates from any single pattern-mixture model may suffer from undercoverage, because uncertainty about model misspecification is not taken into account. In this article, a new class of Bayesian random coefficient pattern-mixture models is developed to address potentially non-ignorable drop-out. Instead of imposing hard equality constraints to overcome inherent inestimability problems in pattern-mixture models, we propose to smooth the polynomial coefficient estimates across patterns using a hierarchical Bayesian model that allows random variation across groups. Using real and simulated data, we show that multiple imputation under a three-level linear mixed-effects model which accommodates a random level due to drop-out groups can be an effective method to deal with non-ignorable drop-out by allowing model uncertainty to be incorporated into the imputation process.     

Marginalized transition models for longitudinal binary data with ignorable and non-ignorable drop-out

We extend the marginalized transition model of Heagerty to accommodate non-ignorable monotone drop-out. Using a selection model, weakly identified drop-out parameters are held constant and their effects evaluated through sensitivity analysis. For data missing at random (MAR), efficiency of inverse probability of censoring weighted generalized estimating equations (IPCW-GEE) is as low as 40 per cent compared to a likelihood-based marginalized transition model (MTM) with comparable modelling burden. MTM and IPCW-GEE regression parameters both display misspecification bias for MAR and non-ignorable missing data, and both reduce bias noticeably by improving model fit.     

Modelling spatial disease rates using maximum likelihood

This paper concerns maximum likelihood estimation for a generalized linear mixed model (GLMM) useful for modelling spatial disease rates. The model allows for log-linear covariate adjustment and local smoothing of rates through estimation of spatially correlated random effects. The covariance structure of the random effects is based on a recently proposed model which parameterizes spatial dependence through the inverse covariance matrix. A Markov chain Monte Carlo algorithm for performing maximum likelihood estimation for this model is described. Results of a computer simulation study that compared maximum likelihood (ML) and penalized quasi-likelihood (PQL) estimators are presented. Compared with PQL, ML produced less biased estimates of the intercept but the ML estimates were slightly more variable. Estimates of the other regression coefficients were unbiased and nearly identical for the two methods. ML estimators of the random effects standard deviation and spatial correlation were more biased than the corresponding PQL estimators. The conclusion is that ML estimators for GLMMs cannot be expected to perform better than PQL for small samples.     

An autoregressive linear mixed effects model for the analysis of longitudinal data which include dropouts and show profiles approaching asymptotes

We are interested in longitudinal data of a continuous response that show profiles with an initial sharp change and approaching asymptotes for each patient, and many patients drop out with a reason related to the response. In this paper, we focus on a model that assumes a dropout process is missing at random (MAR). In this dropout process, we can obtain consistent maximum likelihood estimators as long as both the mean and covariance structures are correctly specified. However, parsimonious covariance structures for the profiles approaching asymptotes are unclear. An autoregressive linear mixed effects model can express the profile with random individual asymptotes. We show that this model provides a new parsimonious covariance structure. The covariance structure at steady state is compound symmetry and the other elements of the covariance depend on the measurement points. In simulation studies, the estimate of the asymptote is unbiased in MAR dropouts, but biased in non-ignorable dropouts. We also applied this model to actual schizophrenia trial data.     

Avoiding bias in mixed model inference for fixed effects

Analysis of a large longitudinal study of children motivated our work. The results illustrate how accurate inference for fixed effects in a general linear mixed model depends on the covariance model selected for the data. Simulation studies have revealed biased inference for the fixed effects with an underspecified covariance structure, at least in small samples. One underspecification common for longitudinal data assumes a simple random intercept and conditional independence of the within-subject errors (i.e., compound symmetry). We prove that the underspecification creates bias in both small and large samples, indicating that recruiting more participants will not alleviate inflation of the Type I error rate associated with fixed effect inference. Enumerations and simulations help quantify the bias and evaluate strategies for avoiding it. When practical, backwards selection of the covariance model, starting with an unstructured pattern, provides the best protection. Tutorial papers can guide the reader in minimizing the chances of falling into the often spurious software trap of nonconvergence. In some cases, the logic of the study design and the scientific context may support a structured pattern, such as an autoregressive structure. The sandwich estimator provides a valid alternative in sufficiently large samples. Authors reporting mixed-model analyses should note possible biases in fixed effects inference because of the following: (i) the covariance model selection process; (ii) the specific covariance model chosen; or (iii) the test approximation.     

Estimation of covariate effects in generalized linear mixed models with a misspecified distribution of random intercepts and slopes

Generalized linear mixed models with random intercepts and slopes provide useful analyses of clustered and longitudinal data and typically require the specification of the distribution of the random effects. Previous work for models with only random intercepts has shown that misspecifying the shape of this distribution may bias estimates of the intercept, but typically leads to little bias in estimates of covariate effects. Very few papers have examined the effects of misspecifying the joint distribution of random intercepts and slopes. However, simulation results in a recent paper suggest that misspecifying the shape of the random slope distribution can yield severely biased estimates of all model parameters. Using analytic results, simulation studies and fits to example data, this paper examines the bias in parameter estimates due to misspecification of the shape of the joint distribution of random intercepts and slopes. Consistent with results for models with only random intercepts, and contrary to the claims of severe bias in a recent paper, we show that misspecification of the joint distribution typically yields little bias in estimates of covariate effects and is restricted to covariates associated with the misspecified random effects distributions. We also show that misspecification of the distribution of random effects has little effect on confidence interval performance. Coverage rates based on the model‐based standard errors from fitted likelihoods were generally quite close to nominal. Copyright © 2012 John Wiley & Sons, Ltd.     

Joint Longitudinal Models for Dealing With Missing at Random Data in Trial-Based Economic Evaluations

ObjectivesIn trial-based economic evaluation, some individuals are typically associated with missing data at some time point, so that their corresponding aggregated outcomes (eg, quality-adjusted life-years) cannot be evaluated. Restricting the analysis to the complete cases is inefficient and can result in biased estimates, while imputation methods are often implemented under a missing at random (MAR) assumption. We propose the use of joint longitudinal models to extend standard approaches by taking into account the longitudinal structure to improve the estimation of the targeted quantities under MAR.MethodsWe compare the results from methods that handle missingness at an aggregated (case deletion, baseline imputation, and joint aggregated models) and disaggregated (joint longitudinal models) level under MAR. The methods are compared using a simulation study and applied to data from 2 real case studies.ResultsSimulations show that, according to which data affect the missingness process, aggregated methods may lead to biased results, while joint longitudinal models lead to valid inferences under MAR. The analysis of the 2 case studies support these results as both parameter estimates and cost-effectiveness results vary based on the amount of data incorporated into the model.ConclusionsOur analyses suggest that methods implemented at the aggregated level are potentially biased under MAR as they ignore the information from the partially observed follow-up data. This limitation can be overcome by extending the analysis to a longitudinal framework using joint models, which can incorporate all the available evidence.     

Methods to Account for Attrition in Longitudinal Data: Do They Work? A Simulation Study

Attrition threatens the internal validity of cohort studies. Epidemiologists use various imputation and weighting methods to limit bias due to attrition. However, the ability of these methods to correct for attrition bias has not been tested. We simulated a cohort of 300 subjects using 500 computer replications to determine whether regression imputation, individual weighting, or multiple imputation is useful to reduce attrition bias. We compared these results to a complete subject analysis. Our logistic regression model included a binary exposure and two confounders. We generated 10, 25, and 40% attrition through three missing data mechanisms: missing completely at random (MCAR), missing at random (MAR) and missing not at random (MNAR), and used four covariance matrices to vary attrition. We compared true and estimated mean odds ratios (ORs), standard deviations (SDs), and coverage. With data MCAR and MAR for all attrition rates, the complete subject analysis produced results at least as valid as those from the imputation and weighting methods. With data MNAR, no method provided unbiased estimates of the OR at attrition rates of 25 or 40%. When observations are not MAR or MCAR, imputation and weighting methods may not effectively reduce attrition bias.     

A determinant‐based criterion for working correlation structure selection in generalized estimating equations

In generalized estimating equations (GEE), the correlation between the repeated observations on a subject is specified with a working correlation matrix. Correct specification of the working correlation structure ensures efficient estimators of the regression coefficients. Among the criteria used, in practice, for selecting working correlation structure, Rotnitzky‐Jewell, Quasi Information Criterion (QIC) and Correlation Information Criterion (CIC) are based on the fact that if the assumed working correlation structure is correct then the model‐based (naive) and the sandwich (robust) covariance estimators of the regression coefficient estimators should be close to each other. The sandwich covariance estimator, used in defining the Rotnitzky‐Jewell, QIC and CIC criteria, is biased downward and has a larger variability than the corresponding model‐based covariance estimator. Motivated by this fact, a new criterion is proposed in this paper based on the bias‐corrected sandwich covariance estimator for selecting an appropriate working correlation structure in GEE. A comparison of the proposed and the competing criteria is shown using simulation studies with correlated binary responses. The results revealed that the proposed criterion generally performs better than the competing criteria. An example of selecting the appropriate working correlation structure has also been shown using the data from Madras Schizophrenia Study. Copyright © 2015 John Wiley & Sons, Ltd.     

Loss to Follow-Up in Cohort Studies: How Much is Too Much?

Loss to follow-up is problematic in most cohort studies and often leads to bias. Although guidelines suggest acceptable follow-up rates, the authors are unaware of studies that test the validity of these recommendations. The objective of this study was to determine whether the recommended follow-up thresholds of 60-80% are associated with biased effects in cohort studies. A simulation study was conducted using 1000 computer replications of a cohort of 500 observations. The logistic regression model included a binary exposure and three confounders. Varied correlation structures of the data represented various levels of confounding. Differing levels of loss to follow-up were generated through three mechanisms: missing completely at random (MCAR), missing at random (MAR) and missing not at random (MNAR). The authors found no important bias with levels of loss that varied from 5 to 60% when loss to follow-up was related to MCAR or MAR mechanisms. However, when observations were lost to follow-up based on a MNAR mechanism, the authors found seriously biased estimates of the odds ratios with low levels of loss to follow-up. Loss to follow-up in cohort studies rarely occurs randomly. Therefore, when planning a cohort study, one should assume that loss to follow-up is MNAR and attempt to achieve the maximum follow-up rate possible.     

Allowing for uncertainty due to missing data in meta-analysis--part 1: two-stage methods

Analysis of a randomized trial with missing outcome data involves untestable assumptions, such as the missing at random (MAR) assumption. Estimated treatment effects are potentially biased if these assumptions are wrong. We quantify the degree of departure from the MAR assumption by the informative missingness odds ratio (IMOR). We incorporate prior beliefs about the IMOR in a Bayesian pattern-mixture model and derive a point estimate and standard error that take account of the uncertainty about the IMOR. In meta-analysis, this model should be used for four separate sensitivity analyses which explore the impact of IMORs that either agree or contrast across trial arms on pooled results via their effects on point estimates or on standard errors. We also propose a variance inflation factor that can be used to assess the influence of trials with many missing outcomes on the meta-analysis. We illustrate the methods using a meta-analysis on psychiatric interventions in deliberate self-harm.     

Analysis of change in the presence of informative censoring: application to a longitudinal clinical trial of progressive renal disease

The rate of change in a continuous variable, measured serially over time, is often used as an outcome in longitudinal studies or clinical trials. When patients terminate the study before the scheduled end of the study, there is a potential for bias in estimation of rate of change using standard methods which ignore the missing data mechanism. These methods include the use of unweighted generalized estimating equations methods and likelihood-based methods assuming an ignorable missing data mechanism. We present a model for analysis of informatively censored data, based on an extension of the two-stage linear random effects model, where each subject's random intercept and slope are allowed to be associated with an underlying time to event. The joint distribution of the continuous responses and the time-to-event variable are then estimated via maximum likelihood using the EM algorithm, and using the bootstrap to calculate standard errors. We illustrate this methodology and compare it to simpler approaches and usual maximum likelihood using data from a multi-centre study of the effects of diet and blood pressure control on progression of renal disease, the Modification of Diet in Renal Disease (MDRD) Study. Sensitivity analyses and simulations are used to evaluate the performance of this methodology in the context of the MDRD data, under various scenarios where the drop-out mechanism is ignorable as well as non-ignorable.     

Adjusting for verification bias in diagnostic test evaluation: a Bayesian approach

Obtaining accurate estimates of the performance of a diagnostic test for some population of patients might be difficult when the sample of subjects used for this purpose is not representative for the whole population. Thus, in the motivating example of this paper a test is evaluated by comparing its results with those given by a gold standard procedure, which yields the disease status verification. However, this procedure is invasive and has a non-negligible risk of serious complications. Moreover, subjects are selected to undergo the gold standard based on some risk factors and the results of the test under study. The test performance estimates based on the selected sample of subjects are biased. This problem was presented in previous studies under the name of verification bias. The current paper introduces a Bayesian method to adjust for this bias, which can be regarded as a missing data problem. In addition, it addresses the case of non-ignorable verification bias. The proposed Bayesian estimation approach provides test performance estimates that are consistent with the results obtained using likelihood-based approach. In addition, the paper studies how valuable the statistical findings are from the perspective of clinical decision making.     

Participants' outcomes gone missing within a network of interventions: Bayesian modeling strategies

Objectives: To investigate the implications of addressing informative missing binary outcome data (MOD) on network meta-analysis (NMA) estimates while applying the missing at random (MAR) assumption under different prior structures of the missingness parameter. Methods: In three motivating examples, we compared six different prior structures of the informative missingness odds ratio (IMOR) parameter in logarithmic scale under pattern-mixture and selection models. Then, we simulated 1000 triangle networks of two-arm trials assuming informative MOD related to interventions. We extended the Bayesian random-effects NMA model for binary outcomes and node-splitting approach to incorporate these 12 models in total. With interval plots, we illustrated the posterior distribution of log OR, common between-trial variance (τ² ), inconsistency factor and probability of being best per intervention under each model. Results: All models gave similar point estimates for all NMA estimates regardless of simulation scenario. For moderate and large MOD, intervention-specific prior structure of log IMOR led to larger posterior standard deviation of log ORs compared to trial-specific and common-within-network prior structures. Hierarchical prior structure led to slightly more precise τ² compared to identical prior structure, particularly for moderate inconsistency and large MOD. Pattern-mixture and selection models agreed for all NMA estimates. Conclusions: Analyzing informative MOD assuming MAR with different prior structures of log IMOR affected mainly the precision of NMA estimates. Reviewers should decide in advance on the prior structure of log IMOR that best aligns with the condition and interventions investigated.     

Robust non‐linear mixed modelling of longitudinal PSA levels after prostate cancer treatment

The objective of this study was to develop a robust non‐linear mixed model for prostate‐specific antigen (PSA) measurements after a high‐intensity focused ultrasound (HIFU) treatment for prostate cancer. The characteristics of these data are the presence of outlying values and non‐normal random effects. A numerical study proved that parameter estimates can be biased if these characteristics are not taken into account. The intra‐patient variability was described by a Student‐t distribution and Dirichlet process priors were assumed for non‐normal random effects; a process that limited the bias and provided more efficient parameter estimates than a classical mixed model with normal residuals and random effects. It was applied to the determination of the best dynamic PSA criterion for the diagnosis of prostate cancer recurrence, but could be used in studies that rely on PSA data to improve prognosis or compare treatment efficiencies and also with other longitudinal biomarkers that, such as PSA, present outlying values and non‐normal random effects. Copyright © 2010 John Wiley & Sons, Ltd.     

A Bayesian approach to prospective binary outcome studies with misclassification in a binary risk factor

Misclassification in a binary exposure variable within an unmatched prospective study may lead to a biased estimate of the disease-exposure relationship. It usually gives falsely small credible intervals because uncertainty in the recorded exposure is not taken into account. When there are several other perfectly measured covariates, interrelationships may introduce further potential for bias. Bayesian methods are proposed for analysing binary outcome studies in which an exposure variable is sometimes misclassified, but its correct values have been validated for a random subsample of the subjects. This Bayesian approach can model relationships between explanatory variables and between exploratory variables and the probabilities of misclassification. Three logistic regressions are used to relate disease to true exposure, misclassified exposure to true exposure and true exposure to other covariates. Credible intervals may be used to make decisions about whether certain parameters are unnecessary and hence whether the model can be reduced in complexity.In the disease-exposure model, for parameters representing coefficients related to perfectly measured covariates, the precision of posterior estimates is only slightly lower than would be found from data with no misclassification. For the risk factor which has misclassification, the estimates of model coefficients obtained are much less biased than those with misclassification ignored.     

Specification of covariance structure in longitudinal data analysis for randomized clinical trials

Misspecification of the covariance structure for repeated measurements in longitudinal analysis may lead to biased estimates of the regression parameters and under or overestimation of the corresponding standard errors in the presence of missing data. The so‐called sandwich estimator can ‘correct’ the variance, but it does not reduce the bias in point estimates. Removing all assumptions from the covariance structure (i.e. using an unstructured (UN) covariance) will remove such biases. However, an excessive amount of missing data may cause convergence problems for iterative algorithms, such as the default Newton–Raphson algorithm in the popular SAS PROC MIXED. This article examines, both through theory and simulations, the existence and the magnitude of these biases. We recommend the use of UN covariance as the default strategy for analyzing longitudinal data from randomized clinical trials with moderate to large number of subjects and small to moderate number of time points. We also present an algorithm to assist in the convergence when the UN covariance is used. Copyright © 2009 John Wiley & Sons, Ltd.     

Out of sight, not out of mind: strategies for handling missing data

OBJECTIVE: To describe and illustrate missing data mechanisms (MCAR, MAR, NMAR) and missing data techniques (MDTs) and offer recommended best practices for addressing missingness. METHOD: We simulated data sets and employed ad hoc MDTs (deletion techniques, mean substitution) and sophisticated MDTs (full information maximum likelihood, Bayesian estimation, multiple imputation) in linear regression analyses. RESULTS: MCAR data yielded unbiased parameter estimates across all MDTs, but loss of power with deletion methods. NMAR results were biased towards larger values and greater significance. Under MAR the sophisticated MDTs returned estimates closer to their original values. CONCLUSION: State-of-the-art, readily available MDTs outperform ad hoc techniques.