A mixed-effects quintile-stratified propensity adjustment for effectiveness analyses of ordered categorical doses

Observational studies can be used to evaluate treatment effectiveness among patients with a broader range of illness severity than typically seen in randomized controlled clinical trials. However, there are several difficulties with observational evaluations including non-equivalent comparison groups, treatment doses and durations that vary widely, and, in longitudinal studies, multiple courses of treatment per subject. A mixed-effects approach to the propensity adjustment for non-equivalent comparison groups is described that can account for each of these perturbations. The strategy involves two stages. First, characteristics that distinguish among subjects who receive various levels of treatment are examined in a model of propensity for treatment intensity using mixed-effects ordinal logistic regression. Second, the propensity-stratified effectiveness of ordered categorical doses is compared in a mixed-effects grouped time survival model of time until recovery. The model is applied in a longitudinal, observational study of antidepressant effectiveness. Then a Monte Carlo simulation study indicates that the strategy has acceptable type I error rates and minimal bias in the estimates of treatment effectiveness. Statistical power exceeds 0.90 for an odds ratio of 1.5 with N = 250 and 500, and is acceptable for an odds ratio of 2.0 with N = 100. Nevertheless, with N = 100, the models that had high intraclass correlation coefficients had greater tendency towards non-convergence. This approach is a useful strategy for observational studies of treatment effectiveness. It is capable of adjusting for selection bias, incorporating multiple observations per subject, and comparing effectiveness of ordinal doses.     

A comparison of mixed-effects quantile stratification propensity adjustment strategies for longitudinal treatment effectiveness analyses of continuous outcomes

The propensity adjustment is used to reduce bias in treatment effectiveness estimates from observational data. We show here that a mixed-effects implementation of the propensity adjustment can reduce bias in longitudinal studies of non-equivalent comparison groups. The strategy examined here involves two stages. Initially, a mixed-effects ordinal logistic regression model of propensity for treatment intensity includes variables that differentiate subjects who receive various doses of time-varying treatments. Second, a mixed-effects linear regression model compares the effectiveness of those ordinal doses on a continuous outcome over time. Here, a simulation study compares bias reduction that is achieved by implementing this propensity adjustment through various forms of stratification. The simulations demonstrate that bias decreased monotonically as the number of quantiles used for stratification increased from two to five. This was particularly pronounced with stronger effects of the confounding variables. The quartile and quintile strategies typically removed in excess of 80-90 per cent of the bias detected in unadjusted models; whereas a median-split approach removed from 20 to 45 per cent of bias. The approach is illustrated in an evaluation of the effectiveness of somatic treatments for major depression in a longitudinal, observational study of affective disorders.     

A dynamic adaptation of the propensity score adjustment for effectiveness analyses of ordinal doses of treatment

The propensity score adjustment is a method to reduce bias in observational studies. We propose a strategy that involves a novel combination of three data analytic techniques, which adapts the propensity adjustment for additional perturbations of longitudinal, observational studies. First, ordinal logistic regression examines propensity for ordinal doses of treatment. Second, a mixed-model approach incorporates the multiple treatment trials and multiple episodes that are characteristic of chronically ill subjects. Finally, a mixed-effects grouped-time survival model incorporates the propensity score in treatment effectiveness analyses. The strategy that is applied here to an observational study of affective illness can also be used to evaluate the effectiveness of treatments for other chronic illnesses.     

Bias associated with using the estimated propensity score as a regression covariate

The use of propensity score methods to adjust for selection bias in observational studies has become increasingly popular in public health and medical research. A substantial portion of studies using propensity score adjustment treat the propensity score as a conventional regression predictor. Through a Monte Carlo simulation study, Austin and colleagues. investigated the bias associated with treatment effect estimation when the propensity score is used as a covariate in nonlinear regression models, such as logistic regression and Cox proportional hazards models. We show that the bias exists even in a linear regression model when the estimated propensity score is used and derive the explicit form of the bias. We also conduct an extensive simulation study to compare the performance of such covariate adjustment with propensity score stratification, propensity score matching, inverse probability of treatment weighted method, and nonparametric functional estimation using splines. The simulation scenarios are designed to reflect real data analysis practice. Instead of specifying a known parametric propensity score model, we generate the data by considering various degrees of overlap of the covariate distributions between treated and control groups. Propensity score matching excels when the treated group is contained within a larger control pool, while the model‐based adjustment may have an edge when treated and control groups do not have too much overlap. Overall, adjusting for the propensity score through stratification or matching followed by regression or using splines, appears to be a good practical strategy. Copyright © 2013 John Wiley & Sons, Ltd.     

Group sequential clinical trials for longitudinal data with analyses using summary statistics

Longitudinal endpoints are used in clinical trials, and the analysis of the results is often conducted using within-individual summary statistics. When these trials are monitored, interim analyses that include subjects with incomplete follow-up can give incorrect decisions due to bias by non-linearity in the true time trajectory of the treatment effect. Linear mixed-effects models can be used to remove this bias, but there is a lack of software to support both the design and implementation of monitoring plans in this setting. This paper considers a clinical trial in which the measurement time schedule is fixed (at least for pre-trial design), and the scientific question is parameterized by a contrast across these measurement times. This setting assures generalizable inference in the presence of non-linear time trajectories. The distribution of the treatment effect estimate at the interim analyses using the longitudinal outcome measurements is given, and software to calculate the amount of information at each interim analysis is provided. The interim information specifies the analysis timing thereby allowing standard group sequential design software packages to be used for trials with longitudinal outcomes. The practical issues with implementation of these designs are described; in particular, methods are presented for consistent estimation of treatment effects at the interim analyses when outcomes are not measured according to the pre-trial schedule. Splus/R functions implementing this inference using appropriate linear mixed-effects models are provided. These designs are illustrated using a clinical trial of statin treatment for the symptoms of peripheral arterial disease.     

Scale of reference bias and the evolution of health

The analysis of subjective measures of well-being-such as self-reports by individuals about their health status is frequently hampered by the problem of scale of reference bias. A particular form of scale of reference bias is age norming. In this study we corrected for scale of reference bias by allowing for individual specific effects in an equation on subjective health. A random effects ordered response model was used to analyze scale of reference bias in self-reported health measures. The results indicate that if we do not control for unobservable individual specific effects, the response to a subjective health state measure suffers from age norming. Age norming can be controlled for by a random effects estimation technique using longitudinal data. Further, estimates are presented on the rate of depreciation of health. Finally, simulations of life expectancy indicate that the estimated model provides a reasonably good fit of the true life expectancy.     

Functional regression analysis using an F test for longitudinal data with large numbers of repeated measures

Longitudinal data sets from certain fields of biomedical research often consist of several variables repeatedly measured on each subject yielding a large number of observations. This characteristic complicates the use of traditional longitudinal modelling strategies, which were primarily developed for studies with a relatively small number of repeated measures per subject. An innovative way to model such 'wide' data is to apply functional regression analysis, an emerging statistical approach in which observations of the same subject are viewed as a sample from a functional space. Shen and Faraway introduced an F test for linear models with functional responses. This paper illustrates how to apply this F test and functional regression analysis to the setting of longitudinal data. A smoking cessation study for methadone-maintained tobacco smokers is analysed for demonstration. In estimating the treatment effects, the functional regression analysis provides meaningful clinical interpretations, and the functional F test provides consistent results supported by a mixed-effects linear regression model. A simulation study is also conducted under the condition of the smoking data to investigate the statistical power for the F test, Wilks' likelihood ratio test, and the linear mixed-effects model using AIC.     

Performance of a propensity score adjustment in longitudinal studies with covariate-dependent representation

Longitudinal observational studies provide rich opportunities to examine treatment effectiveness during the course of a chronic illness. However, there are threats to the validity of observational inferences. For instance, clinician judgment and self‐selection play key roles in treatment assignment. To account for this, an adjustment such as the propensity score can be used if certain assumptions are fulfilled. Here, we consider a problem that could surface in a longitudinal observational study and has been largely overlooked. It can occur when subjects have a varying number of distinct periods of therapeutic intervention. We evaluate the implications of baseline variables in the propensity model being associated with the number of post baseline observations per subject and refer to it as ‘covariate‐dependent representation’. An observational study of antidepressant treatment effectiveness serves as a motivating example. The analyses examine the first 20 years of follow‐up data from the National Institute of Mental Health Collaborative Depression Study, a longitudinal, observational study. A simulation study evaluates the consequences of covariate‐dependent representation in longitudinal observational studies of treatment effectiveness under a range of data specifications.The simulations found that estimates were adversely affected by underrepresentation when there was lower ICC among repeated doses and among repeated outcomes. Copyright © 2012 John Wiley & Sons, Ltd.     

The performance of different propensity‐score methods for estimating differences in proportions (risk differences or absolute risk reductions) in observational studies

Propensity score methods are increasingly being used to estimate the effects of treatments on health outcomes using observational data. There are four methods for using the propensity score to estimate treatment effects: covariate adjustment using the propensity score, stratification on the propensity score, propensity‐score matching, and inverse probability of treatment weighting (IPTW) using the propensity score. When outcomes are binary, the effect of treatment on the outcome can be described using odds ratios, relative risks, risk differences, or the number needed to treat. Several clinical commentators suggested that risk differences and numbers needed to treat are more meaningful for clinical decision making than are odds ratios or relative risks. However, there is a paucity of information about the relative performance of the different propensity‐score methods for estimating risk differences. We conducted a series of Monte Carlo simulations to examine this issue. We examined bias, variance estimation, coverage of confidence intervals, mean‐squared error (MSE), and type I error rates. A doubly robust version of IPTW had superior performance compared with the other propensity‐score methods. It resulted in unbiased estimation of risk differences, treatment effects with the lowest standard errors, confidence intervals with the correct coverage rates, and correct type I error rates. Stratification, matching on the propensity score, and covariate adjustment using the propensity score resulted in minor to modest bias in estimating risk differences. Estimators based on IPTW had lower MSE compared with other propensity‐score methods. Differences between IPTW and propensity‐score matching may reflect that these two methods estimate the average treatment effect and the average treatment effect for the treated, respectively. Copyright © 2010 John Wiley & Sons, Ltd.     

Interval estimation for treatment effects using propensity score matching

In causal studies without random assignment of treatment, causal effects can be estimated using matched treated and control samples, where matches are obtained using estimated propensity scores. Propensity score matching can reduce bias in treatment effect estimators in cases where the matched samples have overlapping covariate distributions. Despite its application in many applied problems, there is no universally employed approach to interval estimation when using propensity score matching. In this article, we present and evaluate approaches to interval estimation when using propensity score matching.     

Comparing longitudinal binary outcomes in an observational oral health study

Observational studies continue to be recognized as viable alternatives to randomized trials when making treatment group comparisons, in spite of drawbacks due mainly to selection bias. Sample selection models have been proposed in the economics literature, and more recently in the medical literature, as a method to adjust for selection bias due to observed and unobserved confounders in observational studies. Application of these models has been limited to cross-sectional observational data and to outcomes that are continuous in nature. In this paper we extend application of these models to include longitudinal studies and binary outcomes. We apply a two-stage probit model using GEE to account for correlated longitudinal binary chewing difficulty outcomes. Chewing difficulty was measured every six months during a 24-month period between two groups of subjects: those either receiving or not receiving dental care. Dental care use was measured at six-month intervals as well. Results from our proposed model are compared to results using a standard GEE model that ignores the potential selection bias introduced by unobserved confounders. In this application, accounting for selection bias made a major difference in the substantive conclusions about the outcomes of interest. This is due in part to an adverse selection phenomenon in which those most in need of treatment (and consequently most likely to benefit from it) are actually the ones least likely to seek treatment. Our application of sample selection models to binary longitudinal observational outcome data should serve as impetus for increased utilization of this promising set of models to other health outcomes studies.     

A varying-coefficient model for the evaluation of time-varying concomitant intervention effects in longitudinal studies

Concomitant interventions are often introduced during a longitudinal clinical trial to patients who respond undesirably to the pre-specified treatments. In addition to the main objective of evaluating the pre-specified treatment effects, an important secondary objective in such a trial is to evaluate whether a concomitant intervention could change a patient's response over time. Because the initiation of a concomitant intervention may depend on the patient's general trend of pre-intervention outcomes, regression approaches that treat the presence of the intervention as a time-dependent covariate may lead to biased estimates for the intervention effects. Borrowing the techniques of Follmann and Wu (Biometrics 1995; 51:151-168) for modeling informative missing data, we propose a varying-coefficient mixed-effects model to evaluate the patient's longitudinal outcome trends before and after the patient's starting time of the intervention. By allowing the random coefficients to be correlated with the patient's starting time of the intervention, our model leads to less biased estimates of the intervention effects. Nonparametric estimation and inferences of the coefficient curves and intervention effects are developed using B-splines. Our methods are demonstrated through a longitudinal clinical trial in depression and heart disease and a simulation study.     

Covariate balance in a Bayesian propensity score analysis of beta blocker therapy in heart failure patients

Regression adjustment for the propensity score is a statistical method that reduces confounding from measured variables in observational data. A Bayesian propensity score analysis extends this idea by using simultaneous estimation of the propensity scores and the treatment effect. In this article, we conduct an empirical investigation of the performance of Bayesian propensity scores in the context of an observational study of the effectiveness of beta-blocker therapy in heart failure patients. We study the balancing properties of the estimated propensity scores. Traditional Frequentist propensity scores focus attention on balancing covariates that are strongly associated with treatment. In contrast, we demonstrate that Bayesian propensity scores can be used to balance the association between covariates and the outcome. This balancing property has the effect of reducing confounding bias because it reduces the degree to which covariates are outcome risk factors.     

A latent variable mixed-effects location scale model that also considers between-person differences in the autocorrelation

In public health research an increasing number of studies is conducted in which intensive longitudinal data is collected in an experience sampling or a daily diary design. Typically, the resulting data is analyzed with a mixed-effects model or mixed-effects location scale model because they allow one to examine a host of interesting longitudinal research questions. Here, we introduce an extension of the mixed-effects location scale model in which measurement error of the observed variables is considered by a latent factor model and in which—in addition to the mean-or location-related effects—the residual variance of the latent factor and the parameters of the autoregressive process of this latent factor can differ between persons. We show how to estimate the parameters of the model with a maximum likelihood approach, whose performance is also compared with a Bayesian approach in a small simulation study. We illustrate the models using a real data example and end with a discussion in which we suggest questions for future research.     

A two-level structural equation model approach for analyzing multivariate longitudinal responses

The analysis of longitudinal data to study changes in variables measured repeatedly over time has received considerable attention in many fields. This paper proposes a two-level structural equation model for analyzing multivariate longitudinal responses that are mixed continuous and ordered categorical variables. The first-level model is defined for measures taken at each time point nested within individuals for investigating their characteristics that are changed with time. The second level is defined for individuals to assess their characteristics that are invariant with time. The proposed model accommodates fixed covariates, nonlinear terms of the latent variables, and missing data. A maximum likelihood (ML) approach is developed for the estimation of parameters and model comparison. Results of a simulation study indicate that the performance of the ML estimation is satisfactory. The proposed methodology is applied to a longitudinal study concerning cocaine use.     

A mixed-effects model for cognitive decline with non-monotone non-response from a two-phase longitudinal study of dementia

Most longitudinal studies of elderly are characterized by substantial drop-out due to death and many other factors beyond the control of the investigators. In a two-phase longitudinal study of dementia, subjects with cognitive impairment skip the first phase survey in the next follow-up, leading to intermittent missing variables measured in that phase. In the context of analysing pre-dementia cognitive decline in an elderly population, both of the two causes of non-response can potentially be informative in the sense that the missingness is dependent on the unobserved outcome. To take these factors into account, mixed-effects models are constructed to allow the outcome and the multiple causes of missing values to share the same 'random parameter' or random effect. The crucial assumption of our model is that the random effects of the model for the outcome and that of the model for the missing-data indicators are linked in a deterministic manner. It can be thought of as an approximation of a more general and realistic situation, in which the two models have distinct, yet dependent, random effects. We conduct a simulation study to investigate possible deviations of the estimates under such a scenario. A second simulation illustrates the magnitude of the bias in estimating the difference of decline rate between two groups when the random effects are linked in different manners for the two groups.     

The impact of missing data on estimation of health-related quality of life outcomes: an analysis of a randomized longitudinal clinical trial

Missing responses for health-related quality of life (HRQL) outcomes are common in clinical trials and may introduce bias as such data are often not missing at random. To evaluate the missingness (dropout) effect when comparing two treatment groups in a longitudinal randomized trial, we analyzed the Functional Assessment of Cancer Therapy Trial Outcome Index (TOI) change over 12 months for newly diagnosed patients with chronic myeloid leukemia. HRQL assessment was expected at baseline and months 1, 2, 3, 4, 5, 6, 9 and 12. We defined completers as those with baseline and month 12 TOI, and dropouts as all others as long as they had a baseline score. We defined censoring time as the time interval between baseline and the scheduled month 12 visit dates and approximate time-to-dropout as the time interval from baseline to the midpoint between date of the last reported TOI and the scheduled next visit date. A mixed-effects model was first built to assess treatment effect; a pattern-mixture model and a joint model were then built to account for non-ignorable dropout. Intermittent missing data were assumed to be missing at random. A square root transformation of TOI scores was taken to fulfill the normality and homogeneity assumption at each time point in all the models. The mixed-effects model revealed significant (P < 0.001) between-group differences at each visit except for baseline. The joint model generated similar parameter estimates as the separate longitudinal and survival sub-models with a significant association parameter (P = 0.039) indicating negative association between slope of TOI and hazard of dropout and thus non-ignorable dropout. The pattern-mixture model parameter estimates were fairly similar to those generated from the joint model. When non-ignorable missing data exist in longitudinal studies, a joint model is useful to quantify the relationship between dropout and outcome. In addition, it is important to examine underlying assumptions and utilize multiple missing data models including the pattern mixture model to assess sensitivity of model based inference to assumptions about missing mechanisms.     

Estimation of propensity scores using generalized additive models

Propensity score matching is often used in observational studies to create treatment and control groups with similar distributions of observed covariates. Typically, propensity scores are estimated using logistic regressions that assume linearity between the logistic link and the predictors. We evaluate the use of generalized additive models (GAMs) for estimating propensity scores. We compare logistic regressions and GAMs in terms of balancing covariates using simulation studies with artificial and genuine data. We find that, when the distributions of covariates in the treatment and control groups overlap sufficiently, using GAMs can improve overall covariate balance, especially for higher-order moments of distributions. When the distributions in the two groups overlap insufficiently, GAM more clearly reveals this fact than logistic regression does. We also demonstrate via simulation that matching with GAMs can result in larger reductions in bias when estimating treatment effects than matching with logistic regression.     

The performance of different propensity score methods for estimating marginal hazard ratios

Propensity score methods are increasingly being used to reduce or minimize the effects of confounding when estimating the effects of treatments, exposures, or interventions when using observational or non‐randomized data. Under the assumption of no unmeasured confounders, previous research has shown that propensity score methods allow for unbiased estimation of linear treatment effects (e.g., differences in means or proportions). However, in biomedical research, time‐to‐event outcomes occur frequently. There is a paucity of research into the performance of different propensity score methods for estimating the effect of treatment on time‐to‐event outcomes. Furthermore, propensity score methods allow for the estimation of marginal or population‐average treatment effects. We conducted an extensive series of Monte Carlo simulations to examine the performance of propensity score matching (1:1 greedy nearest‐neighbor matching within propensity score calipers), stratification on the propensity score, inverse probability of treatment weighting (IPTW) using the propensity score, and covariate adjustment using the propensity score to estimate marginal hazard ratios. We found that both propensity score matching and IPTW using the propensity score allow for the estimation of marginal hazard ratios with minimal bias. Of these two approaches, IPTW using the propensity score resulted in estimates with lower mean squared error when estimating the effect of treatment in the treated. Stratification on the propensity score and covariate adjustment using the propensity score result in biased estimation of both marginal and conditional hazard ratios. Applied researchers are encouraged to use propensity score matching and IPTW using the propensity score when estimating the relative effect of treatment on time‐to‐event outcomes. Copyright © 2012 John Wiley & Sons, Ltd.     

A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: a Monte Carlo study

The propensity score--the probability of exposure to a specific treatment conditional on observed variables--is increasingly being used in observational studies. Creating strata in which subjects are matched on the propensity score allows one to balance measured variables between treated and untreated subjects. There is an ongoing controversy in the literature as to which variables to include in the propensity score model. Some advocate including those variables that predict treatment assignment, while others suggest including all variables potentially related to the outcome, and still others advocate including only variables that are associated with both treatment and outcome. We provide a case study of the association between drug exposure and mortality to show that including a variable that is related to treatment, but not outcome, does not improve balance and reduces the number of matched pairs available for analysis. In order to investigate this issue more comprehensively, we conducted a series of Monte Carlo simulations of the performance of propensity score models that contained variables related to treatment allocation, or variables that were confounders for the treatment-outcome pair, or variables related to outcome or all variables related to either outcome or treatment or neither. We compared the use of these different propensity scores models in matching and stratification in terms of the extent to which they balanced variables. We demonstrated that all propensity scores models balanced measured confounders between treated and untreated subjects in a propensity-score matched sample. However, including only the true confounders or the variables predictive of the outcome in the propensity score model resulted in a substantially larger number of matched pairs than did using the treatment-allocation model. Stratifying on the quintiles of any propensity score model resulted in residual imbalance between treated and untreated subjects in the upper and lower quintiles. Greater balance between treated and untreated subjects was obtained after matching on the propensity score than after stratifying on the quintiles of the propensity score. When a confounding variable was omitted from any of the propensity score models, then matching or stratifying on the propensity score resulted in residual imbalance in prognostically important variables between treated and untreated subjects. We considered four propensity score models for estimating treatment effects: the model that included only true confounders; the model that included all variables associated with the outcome; the model that included all measured variables; and the model that included all variables associated with treatment selection. Reduction in bias when estimating a null treatment effect was equivalent for all four propensity score models when propensity score matching was used. Reduction in bias was marginally greater for the first two propensity score models than for the last two propensity score models when stratification on the quintiles of the propensity score model was employed. Furthermore, omitting a confounding variable from the propensity score model resulted in biased estimation of the treatment effect. Finally, the mean squared error for estimating a null treatment effect was lower when either of the first two propensity scores was used compared to when either of the last two propensity score models was used.