A tutorial on propensity score estimation for multiple treatments using generalized boosted models

The use of propensity scores to control for pretreatment imbalances on observed variables in non‐randomized or observational studies examining the causal effects of treatments or interventions has become widespread over the past decade. For settings with two conditions of interest such as a treatment and a control, inverse probability of treatment weighted estimation with propensity scores estimated via boosted models has been shown in simulation studies to yield causal effect estimates with desirable properties. There are tools (e.g., the twang package in R) and guidance for implementing this method with two treatments. However, there is not such guidance for analyses of three or more treatments. The goals of this paper are twofold: (1) to provide step‐by‐step guidance for researchers who want to implement propensity score weighting for multiple treatments and (2) to propose the use of generalized boosted models (GBM) for estimation of the necessary propensity score weights. We define the causal quantities that may be of interest to studies of multiple treatments and derive weighted estimators of those quantities. We present a detailed plan for using GBM to estimate propensity scores and using those scores to estimate weights and causal effects. We also provide tools for assessing balance and overlap of pretreatment variables among treatment groups in the context of multiple treatments. A case study examining the effects of three treatment programs for adolescent substance abuse demonstrates the methods. Copyright © 2013 John Wiley & Sons, Ltd.     

Generalized propensity score for estimating the average treatment effect of multiple treatments

The propensity score method is widely used in clinical studies to estimate the effect of a treatment with two levels on patient's outcomes. However, due to the complexity of many diseases, an effective treatment often involves multiple components. For example, in the practice of Traditional Chinese Medicine (TCM), an effective treatment may include multiple components, e.g. Chinese herbs, acupuncture, and massage therapy. In clinical trials involving TCM, patients could be randomly assigned to either the treatment or control group, but they or their doctors may make different choices about which treatment component to use. As a result, treatment components are not randomly assigned. Rosenbaum and Rubin proposed the propensity score method for binary treatments, and Imbens extended their work to multiple treatments. These authors defined the generalized propensity score as the conditional probability of receiving a particular level of the treatment given the pre-treatment variables. In the present work, we adopted this approach and developed a statistical methodology based on the generalized propensity score in order to estimate treatment effects in the case of multiple treatments. Two methods were discussed and compared: propensity score regression adjustment and propensity score weighting. We used these methods to assess the relative effectiveness of individual treatments in the multiple-treatment IMPACT clinical trial. The results reveal that both methods perform well when the sample size is moderate or large.     

Addressing missing data in confounders when estimating propensity scores for continuous exposures

Propensity score models are frequently used to estimate causal effects in observational studies. One unresolved issue in fitting these models is handling missing values in the propensity score model covariates. As these models usually contain a large set of covariates, using only individuals with complete data significantly decreases the sample size and statistical power. Several missing data imputation approaches have been proposed, including multiple imputation (MI), MI with missingness pattern (MIMP), and treatment mean imputation. Generalized boosted modeling (GBM), which is a nonparametric approach to estimate propensity scores, can automatically handle missingness in the covariates. Although the performance of MI, MIMP, and treatment mean imputation have previously been compared for binary treatments, they have not been compared for continuous exposures or with single imputation and GBM. We compared these approaches in estimating the generalized propensity score (GPS) for a continuous exposure in both a simulation study and in empirical data. Using GBM with the incomplete data to estimate the GPS did not perform well in the simulation. Missing values should be imputed before estimating propensity scores using GBM or any other approach for estimating the GPS.     

Multiple imputation procedures for estimating causal effects with multiple treatments with application to the comparison of healthcare providers

Choosing between multiple healthcare providers requires us to simultaneously compare the expected outcomes under each provider. This comparison is complex because the composition of patients treated by each provider may differ. Similar issues arise when simultaneously comparing the adverse effects of interventions using non-randomized data. To simultaneously estimate the effects of multiple providers/interventions we propose procedures that explicitly impute the set of potential outcomes for each subject. The procedures are based on different specifications of the generalized additive models (GAM) and the Bayesian additive regression trees (BART). We compare the performance of the proposed procedures to previously proposed matching and weighting procedures using an extensive simulation study for continuous outcomes. Our simulations show that when the distributions of the covariates across treatment groups have adequate overlap, the multiple imputation procedures based on separate BART or GAM models in each treatment group are generally superior to weighting based methods and have similar and sometimes better performance than matching on the logit of the generalized propensity score. Another advantage of these multiple imputation procedures is the ability to provide point and interval estimates to a wide range of causal effect estimands. We apply the proposed procedures to comparing multiple nursing homes in Massachusetts for readmission outcomes. The proposed approach can be applied to other causal effects applications with multiple treatments.     

Conditioning on the propensity score can result in biased estimation of common measures of treatment effect: a Monte Carlo study

Propensity score methods are increasingly being used to estimate causal treatment effects in the medical literature. Conditioning on the propensity score results in unbiased estimation of the expected difference in observed responses to two treatments. The degree to which conditioning on the propensity score introduces bias into the estimation of the conditional odds ratio or conditional hazard ratio, which are frequently used as measures of treatment effect in observational studies, has not been extensively studied. We conducted Monte Carlo simulations to determine the degree to which propensity score matching, stratification on the quintiles of the propensity score, and covariate adjustment using the propensity score result in biased estimation of conditional odds ratios, hazard ratios, and rate ratios. We found that conditioning on the propensity score resulted in biased estimation of the true conditional odds ratio and the true conditional hazard ratio. In all scenarios examined, treatment effects were biased towards the null treatment effect. However, conditioning on the propensity score did not result in biased estimation of the true conditional rate ratio. In contrast, conventional regression methods allowed unbiased estimation of the true conditional treatment effect when all variables associated with the outcome were included in the regression model. The observed bias in propensity score methods is due to the fact that regression models allow one to estimate conditional treatment effects, whereas propensity score methods allow one to estimate marginal treatment effects. In several settings with non-linear treatment effects, marginal and conditional treatment effects do not coincide.     

On the use of discrete choice models for causal inference

Methodology for causal inference based on propensity scores has been developed and popularized in the last two decades. However, the majority of the methodology has concentrated on binary treatments. Only recently have these methods been extended to settings with multi-valued treatments. We propose a number of discrete choice models for estimating the propensity scores. The models differ in terms of flexibility with respect to potential correlation between treatments, and, in turn, the accuracy of the estimated propensity scores. We present the effects of discrete choice models used on performance of the causal estimators through a Monte Carlo study. We also illustrate the use of discrete choice models to estimate the effect of antipsychotic drug use on the risk of diabetes in a cohort of adults with schizophrenia.     

The use of propensity scores to evaluate outcomes for community clinics

This article presents a model for mental health programs to estimate causal effects of treatment in community settings, where experimental studies, in which subjects are randomly assigned to treatment and control groups, are not feasible. This article describes an observational study that used a propensity score analysis with stratification and a repeated-measures analysis of covariance model to estimate treatment effects. This article includes results from one example site that was identified as having an exceptional home-based community program. The results include treatment effects for 3 outcomes identified as useful goals for home-based community programs. The study also serves as a model of how local programs can establish credibility where no evidence-based treatments exist for severely impaired youths.     

Simultaneous record linkage and causal inference with propensity score subclassification

We develop methodology for causal inference in observational studies when using propensity score subclassification on data constructed with probabilistic record linkage techniques. We focus on scenarios where covariates and binary treatment assignments are in one file and outcomes are in another file, and the goal is to estimate an additive treatment effect by merging the files. We assume that the files can be linked using variables common to both files, eg, names or birth dates, but that links are subject to errors, eg, due to reporting errors in the linking variables. We develop methodology for cases where such reporting errors are independent of the other variables on the files. We describe conceptually how linkage errors can affect causal estimates in subclassification contexts. We also present and evaluate several algorithms for deciding which record pairs to use in estimation of causal effects. Using simulation studies, we demonstrate that case selection procedures can result in improved accuracy in estimates of treatment effects from linked data compared to using only cases known to be true links.     

The performance of different propensity‐score methods for estimating differences in proportions (risk differences or absolute risk reductions) in observational studies

Propensity score methods are increasingly being used to estimate the effects of treatments on health outcomes using observational data. There are four methods for using the propensity score to estimate treatment effects: covariate adjustment using the propensity score, stratification on the propensity score, propensity‐score matching, and inverse probability of treatment weighting (IPTW) using the propensity score. When outcomes are binary, the effect of treatment on the outcome can be described using odds ratios, relative risks, risk differences, or the number needed to treat. Several clinical commentators suggested that risk differences and numbers needed to treat are more meaningful for clinical decision making than are odds ratios or relative risks. However, there is a paucity of information about the relative performance of the different propensity‐score methods for estimating risk differences. We conducted a series of Monte Carlo simulations to examine this issue. We examined bias, variance estimation, coverage of confidence intervals, mean‐squared error (MSE), and type I error rates. A doubly robust version of IPTW had superior performance compared with the other propensity‐score methods. It resulted in unbiased estimation of risk differences, treatment effects with the lowest standard errors, confidence intervals with the correct coverage rates, and correct type I error rates. Stratification, matching on the propensity score, and covariate adjustment using the propensity score resulted in minor to modest bias in estimating risk differences. Estimators based on IPTW had lower MSE compared with other propensity‐score methods. Differences between IPTW and propensity‐score matching may reflect that these two methods estimate the average treatment effect and the average treatment effect for the treated, respectively. Copyright © 2010 John Wiley & Sons, Ltd.     

针对连续型处理因素的广义倾向性评分估计方法简介

在观察性研究中进行因果推断的众多方法中,用于控制已测量混杂的倾向性评分方法应用越来越广泛。该类方法主要分为两步：首先估计倾向性评分,然后采取回归、加权、匹配和分层等手段进一步估计感兴趣的因果参数。不同于传统的二分类处理情况,近年来针对连续型处理因素的广义倾向性评分方法被提出。目前已发展出了许多估计广义倾向性评分和直接估...     

On the joint use of propensity and prognostic scores in estimation of the average treatment effect on the treated: a simulation study

Propensity and prognostic score methods seek to improve the quality of causal inference in non‐randomized or observational studies by replicating the conditions found in a controlled experiment, at least with respect to observed characteristics. Propensity scores model receipt of the treatment of interest; prognostic scores model the potential outcome under a single treatment condition. While the popularity of propensity score methods continues to grow, prognostic score methods and methods combining propensity and prognostic scores have thus far received little attention. To this end, we performed a simulation study that compared subclassification and full matching on a single estimated propensity or prognostic score with three approaches combining the estimated propensity and prognostic scores: full matching on a Mahalanobis distance combining the estimated propensity and prognostic scores (FULL–MAHAL); full matching on the estimated prognostic propensity score within propensity score calipers (FULL–PGPPTY); and subclassification on an estimated propensity and prognostic score grid with 5 × 5 subclasses (SUBCLASS(5*5)). We considered settings in which one, both, or neither score model was misspecified. The data generating mechanisms varied in the degree of linearity and additivity in the true treatment assignment and outcome models. FULL–MAHAL and FULL–PGPPTY exhibited strong to superior performance in root mean square error terms across all simulation settings and scenarios. Methods combining propensity and prognostic scores were no less robust to model misspecification than single‐score methods even when both score models were incorrectly specified. Our findings support the joint use of propensity and prognostic scores in estimation of the average treatment effect on the treated. Copyright © 2013 John Wiley & Sons, Ltd.     

A novel approach for propensity score matching and stratification for multiple treatments: Application to an electronic health record–derived study

Currently, methods for conducting multiple treatment propensity scoring in the presence of high-dimensional covariate spaces that result from “big data” are lacking—the most prominent method relies on inverse probability treatment weighting (IPTW). However, IPTW only utilizes one element of the generalized propensity score (GPS) vector, which can lead to a loss of information and inadequate covariate balance in the presence of multiple treatments. This limitation motivates the development of a novel propensity score method that uses the entire GPS vector to establish a scalar balancing score that, when adjusted for, achieves covariate balance in the presence of potentially high-dimensional covariates. Specifically, the generalized propensity score cumulative distribution function (GPS-CDF) method is introduced. A one-parameter power function fits the CDF of the GPS vector and a resulting scalar balancing score is used for matching and/or stratification. Simulation results show superior performance of the new method compared to IPTW both in achieving covariate balance and estimating average treatment effects in the presence of multiple treatments. The proposed approach is applied to a study derived from electronic medical records to determine the causal relationship between three different vasopressors and mortality in patients with non-traumatic aneurysmal subarachnoid hemorrhage. Results suggest that the GPS-CDF method performs well when applied to large observational studies with multiple treatments that have large covariate spaces.     

The use of the propensity score for estimating treatment effects: administrative versus clinical data

There is an increasing interest in using administrative data to estimate the treatment effects of interventions. While administrative data are relatively inexpensive to obtain and provide population coverage, they are frequently characterized by lack of clinical detail, often leading to problematic confounding when they are used to conduct observational research. Propensity score methods are increasingly being used to address confounding in estimating the effects of interventions in such studies. Using data on patients discharged from hospital for whom both administrative data and detailed clinical data obtained from chart reviews were available, we examined the degree to which stratifying on the quintiles of propensity scores derived from administrative data was able to balance patient characteristics measured in clinical data. We also determined the extent to which measures of treatment effect obtained using propensity score methods were similar to those obtained using traditional regression methods. As a test case, we examined the treatment effects of ASA and beta-blockers following acute myocardial infarction. We demonstrated that propensity scores developed using administrative data do not necessarily balance patient characteristics contained in clinical data. Furthermore, measures of treatment effectiveness were attenuated when obtained using clinical data compared to when administrative data were used.     

The performance of different propensity score methods for estimating marginal hazard ratios

Propensity score methods are increasingly being used to reduce or minimize the effects of confounding when estimating the effects of treatments, exposures, or interventions when using observational or non‐randomized data. Under the assumption of no unmeasured confounders, previous research has shown that propensity score methods allow for unbiased estimation of linear treatment effects (e.g., differences in means or proportions). However, in biomedical research, time‐to‐event outcomes occur frequently. There is a paucity of research into the performance of different propensity score methods for estimating the effect of treatment on time‐to‐event outcomes. Furthermore, propensity score methods allow for the estimation of marginal or population‐average treatment effects. We conducted an extensive series of Monte Carlo simulations to examine the performance of propensity score matching (1:1 greedy nearest‐neighbor matching within propensity score calipers), stratification on the propensity score, inverse probability of treatment weighting (IPTW) using the propensity score, and covariate adjustment using the propensity score to estimate marginal hazard ratios. We found that both propensity score matching and IPTW using the propensity score allow for the estimation of marginal hazard ratios with minimal bias. Of these two approaches, IPTW using the propensity score resulted in estimates with lower mean squared error when estimating the effect of treatment in the treated. Stratification on the propensity score and covariate adjustment using the propensity score result in biased estimation of both marginal and conditional hazard ratios. Applied researchers are encouraged to use propensity score matching and IPTW using the propensity score when estimating the relative effect of treatment on time‐to‐event outcomes. Copyright © 2012 John Wiley & Sons, Ltd.     

A Bootstrap Confidence Interval Procedure for the Treatment Effect Using Propensity Score Subclassification

In the analysis of observational studies, propensity score subclassification has been shown to be a powerful method for adjusting unbalanced covariates for the purpose of causal inferences. One practical difficulty in carrying out such an analysis is to obtain a correct variance estimate for inference, while reducing bias in the estimate of the treatment effect due to an imbalance in the measured covariates. In this paper, we propose a bootstrap procedure for the inferences concerning the average treatment effect; our bootstrap method is based on an extension of Efron's bias-corrected accelerated (BCa) bootstrap confidence interval to a two-sample problem. Unlike the currently available inference procedures based on propensity score subclassifications, the validity of the proposed method does not rely on a particular form of variance estimation. A brief simulation study is included to evaluate the operating characteristics of the proposed procedure. We conclude the paper by illustrating the new procedure through a clinical application comparing the renal effects of two non-steroidal anti-inammatory drugs (NSAIDs).     

On the use of propensity scores in principal causal effect estimation

We examine the practicality of propensity score methods for estimating causal treatment effects conditional on intermediate posttreatment outcomes (principal effects) in the context of randomized experiments. In particular, we focus on the sensitivity of principal causal effect estimates to violation of principal ignorability, which is the primary assumption that underlies the use of propensity score methods to estimate principal effects. Under principal ignorability (PI), principal strata membership is conditionally independent of the potential outcome under control given the pre‐treatment covariates; i.e. there are no differences in the potential outcomes under control across principal strata given the observed pretreatment covariates. Under this assumption, principal scores modeling principal strata membership can be estimated based solely on the observed covariates and used to predict strata membership and estimate principal effects. While this assumption underlies the use of propensity scores in this setting, sensitivity to violations of it has not been studied rigorously. In this paper, we explicitly define PI using the outcome model (although we do not actually use this outcome model in estimating principal scores) and systematically examine how deviations from the assumption affect estimates, including how the strength of association between principal stratum membership and covariates modifies the performance. We find that when PI is violated, very strong covariate predictors of stratum membership are needed to yield accurate estimates of principal effects. Copyright © 2009 John Wiley & Sons, Ltd.     

倾向得分区间匹配法用于非随机对照试验的探索与研究

目的探索研究倾向得分区间匹配法在非随机对照试验中用于均衡组间混杂因素的能力,并与logistic回归分析方法和倾向得分卡钳匹配进行比较。方法通过Monte Carlo模拟分析倾向得分区间匹配法处理二分类资料的能力,并与传统的logistic回归方法以及倾向得分卡钳匹配法进行比较,通过I类错误、检验效能、标准化差异以及匹配比例等指标进行综合评价。结果倾向得分区间匹配法与logistic回归法以及倾向得分卡钳匹配法的检验效能、I类错误、标准化差异和匹配比例四个评价指标无明显差异。结论在观察性研究和流行病学研究中,采用倾向得分区间匹配法均衡组间协变量得到真实的处理效应具有很高的实用价值。     

Estimation of Causal Effects using Propensity Score Weighting: An Application to Data on Right Heart Catheterization

We consider methods for estimating causal effects of treatments when treatment assignment is unconfounded with outcomes conditional on a possibly large set of covariates. Robins and Rotnitzky (1995) suggested combining regression adjustment with weighting based on the propensity score (Rosenbaum and Rubin, 1983). We adopt this approach, allowing for a flexible specification of both the propensity score and the regression function. We apply these methods to data on the effects of right heart catheterization (RHC) studied in Connors et al (1996), and we find that our estimator gives stable estimates over a wide range of values for the two parameters governing the selection of variables.     

Interval estimation for treatment effects using propensity score matching

In causal studies without random assignment of treatment, causal effects can be estimated using matched treated and control samples, where matches are obtained using estimated propensity scores. Propensity score matching can reduce bias in treatment effect estimators in cases where the matched samples have overlapping covariate distributions. Despite its application in many applied problems, there is no universally employed approach to interval estimation when using propensity score matching. In this article, we present and evaluate approaches to interval estimation when using propensity score matching.     

A comparison of propensity score methods: a case-study estimating the effectiveness of post-AMI statin use

There is an increasing interest in the use of propensity score methods to estimate causal effects in observational studies. However, recent systematic reviews have demonstrated that propensity score methods are inconsistently used and frequently poorly applied in the medical literature. In this study, we compared the following propensity score methods for estimating the reduction in all-cause mortality due to statin therapy for patients hospitalized with acute myocardial infarction: propensity-score matching, stratification using the propensity score, covariate adjustment using the propensity score, and weighting using the propensity score. We used propensity score methods to estimate both adjusted treated effects and the absolute and relative risk reduction in all-cause mortality. We also examined the use of statistical hypothesis testing, standardized differences, box plots, non-parametric density estimates, and quantile-quantile plots to assess residual confounding that remained after stratification or matching on the propensity score. Estimates of the absolute reduction in 3-year mortality ranged from 2.1 to 4.5 per cent, while estimates of the relative risk reduction ranged from 13.3 to 17.0 per cent. Adjusted estimates of the reduction in the odds of 3-year death varied from 15 to 24 per cent across the different propensity score methods.