The use of prognostic scores for causal inference with general treatment regimes

In nonrandomised studies, inferring causal effects requires appropriate methods for addressing confounding bias. Although it is common to adopt propensity score analysis to this purpose, prognostic score analysis has recently been proposed as an alternative strategy. While both approaches were originally introduced to estimate causal effects for binary interventions, the theory of propensity score has since been extended to the case of general treatment regimes. Indeed, many treatments are not assigned in a binary fashion and require a certain extent of dosing. Hence, researchers may often be interested in estimating treatment effects across multiple exposures. To the best of our knowledge, the prognostic score analysis has not been yet generalised to this case. In this article, we describe the theory of prognostic scores for causal inference with general treatment regimes. Our methods can be applied to compare multiple treatments using nonrandomised data, a topic of great relevance in contemporary evaluations of clinical interventions. We propose estimators for the average treatment effects in different populations of interest, the validity of which is assessed through a series of simulations. Finally, we present an illustrative case in which we estimate the effect of the delay to Aspirin administration on a composite outcome of death or dependence at 6 months in stroke patients.     

A tutorial on propensity score estimation for multiple treatments using generalized boosted models

The use of propensity scores to control for pretreatment imbalances on observed variables in non‐randomized or observational studies examining the causal effects of treatments or interventions has become widespread over the past decade. For settings with two conditions of interest such as a treatment and a control, inverse probability of treatment weighted estimation with propensity scores estimated via boosted models has been shown in simulation studies to yield causal effect estimates with desirable properties. There are tools (e.g., the twang package in R) and guidance for implementing this method with two treatments. However, there is not such guidance for analyses of three or more treatments. The goals of this paper are twofold: (1) to provide step‐by‐step guidance for researchers who want to implement propensity score weighting for multiple treatments and (2) to propose the use of generalized boosted models (GBM) for estimation of the necessary propensity score weights. We define the causal quantities that may be of interest to studies of multiple treatments and derive weighted estimators of those quantities. We present a detailed plan for using GBM to estimate propensity scores and using those scores to estimate weights and causal effects. We also provide tools for assessing balance and overlap of pretreatment variables among treatment groups in the context of multiple treatments. A case study examining the effects of three treatment programs for adolescent substance abuse demonstrates the methods. Copyright © 2013 John Wiley & Sons, Ltd.     

A novel approach for propensity score matching and stratification for multiple treatments: Application to an electronic health record–derived study

Currently, methods for conducting multiple treatment propensity scoring in the presence of high-dimensional covariate spaces that result from “big data” are lacking—the most prominent method relies on inverse probability treatment weighting (IPTW). However, IPTW only utilizes one element of the generalized propensity score (GPS) vector, which can lead to a loss of information and inadequate covariate balance in the presence of multiple treatments. This limitation motivates the development of a novel propensity score method that uses the entire GPS vector to establish a scalar balancing score that, when adjusted for, achieves covariate balance in the presence of potentially high-dimensional covariates. Specifically, the generalized propensity score cumulative distribution function (GPS-CDF) method is introduced. A one-parameter power function fits the CDF of the GPS vector and a resulting scalar balancing score is used for matching and/or stratification. Simulation results show superior performance of the new method compared to IPTW both in achieving covariate balance and estimating average treatment effects in the presence of multiple treatments. The proposed approach is applied to a study derived from electronic medical records to determine the causal relationship between three different vasopressors and mortality in patients with non-traumatic aneurysmal subarachnoid hemorrhage. Results suggest that the GPS-CDF method performs well when applied to large observational studies with multiple treatments that have large covariate spaces.     

The performance of different propensity‐score methods for estimating differences in proportions (risk differences or absolute risk reductions) in observational studies

Propensity score methods are increasingly being used to estimate the effects of treatments on health outcomes using observational data. There are four methods for using the propensity score to estimate treatment effects: covariate adjustment using the propensity score, stratification on the propensity score, propensity‐score matching, and inverse probability of treatment weighting (IPTW) using the propensity score. When outcomes are binary, the effect of treatment on the outcome can be described using odds ratios, relative risks, risk differences, or the number needed to treat. Several clinical commentators suggested that risk differences and numbers needed to treat are more meaningful for clinical decision making than are odds ratios or relative risks. However, there is a paucity of information about the relative performance of the different propensity‐score methods for estimating risk differences. We conducted a series of Monte Carlo simulations to examine this issue. We examined bias, variance estimation, coverage of confidence intervals, mean‐squared error (MSE), and type I error rates. A doubly robust version of IPTW had superior performance compared with the other propensity‐score methods. It resulted in unbiased estimation of risk differences, treatment effects with the lowest standard errors, confidence intervals with the correct coverage rates, and correct type I error rates. Stratification, matching on the propensity score, and covariate adjustment using the propensity score resulted in minor to modest bias in estimating risk differences. Estimators based on IPTW had lower MSE compared with other propensity‐score methods. Differences between IPTW and propensity‐score matching may reflect that these two methods estimate the average treatment effect and the average treatment effect for the treated, respectively. Copyright © 2010 John Wiley & Sons, Ltd.     

针对连续型处理因素的广义倾向性评分估计方法简介

在观察性研究中进行因果推断的众多方法中,用于控制已测量混杂的倾向性评分方法应用越来越广泛。该类方法主要分为两步：首先估计倾向性评分,然后采取回归、加权、匹配和分层等手段进一步估计感兴趣的因果参数。不同于传统的二分类处理情况,近年来针对连续型处理因素的广义倾向性评分方法被提出。目前已发展出了许多估计广义倾向性评分和直接估...     

Addressing missing data in confounders when estimating propensity scores for continuous exposures

Propensity score models are frequently used to estimate causal effects in observational studies. One unresolved issue in fitting these models is handling missing values in the propensity score model covariates. As these models usually contain a large set of covariates, using only individuals with complete data significantly decreases the sample size and statistical power. Several missing data imputation approaches have been proposed, including multiple imputation (MI), MI with missingness pattern (MIMP), and treatment mean imputation. Generalized boosted modeling (GBM), which is a nonparametric approach to estimate propensity scores, can automatically handle missingness in the covariates. Although the performance of MI, MIMP, and treatment mean imputation have previously been compared for binary treatments, they have not been compared for continuous exposures or with single imputation and GBM. We compared these approaches in estimating the generalized propensity score (GPS) for a continuous exposure in both a simulation study and in empirical data. Using GBM with the incomplete data to estimate the GPS did not perform well in the simulation. Missing values should be imputed before estimating propensity scores using GBM or any other approach for estimating the GPS.     

Conditioning on the propensity score can result in biased estimation of common measures of treatment effect: a Monte Carlo study

Propensity score methods are increasingly being used to estimate causal treatment effects in the medical literature. Conditioning on the propensity score results in unbiased estimation of the expected difference in observed responses to two treatments. The degree to which conditioning on the propensity score introduces bias into the estimation of the conditional odds ratio or conditional hazard ratio, which are frequently used as measures of treatment effect in observational studies, has not been extensively studied. We conducted Monte Carlo simulations to determine the degree to which propensity score matching, stratification on the quintiles of the propensity score, and covariate adjustment using the propensity score result in biased estimation of conditional odds ratios, hazard ratios, and rate ratios. We found that conditioning on the propensity score resulted in biased estimation of the true conditional odds ratio and the true conditional hazard ratio. In all scenarios examined, treatment effects were biased towards the null treatment effect. However, conditioning on the propensity score did not result in biased estimation of the true conditional rate ratio. In contrast, conventional regression methods allowed unbiased estimation of the true conditional treatment effect when all variables associated with the outcome were included in the regression model. The observed bias in propensity score methods is due to the fact that regression models allow one to estimate conditional treatment effects, whereas propensity score methods allow one to estimate marginal treatment effects. In several settings with non-linear treatment effects, marginal and conditional treatment effects do not coincide.     

The use of propensity scores to evaluate outcomes for community clinics

This article presents a model for mental health programs to estimate causal effects of treatment in community settings, where experimental studies, in which subjects are randomly assigned to treatment and control groups, are not feasible. This article describes an observational study that used a propensity score analysis with stratification and a repeated-measures analysis of covariance model to estimate treatment effects. This article includes results from one example site that was identified as having an exceptional home-based community program. The results include treatment effects for 3 outcomes identified as useful goals for home-based community programs. The study also serves as a model of how local programs can establish credibility where no evidence-based treatments exist for severely impaired youths.     

The performance of different propensity score methods for estimating marginal hazard ratios

Propensity score methods are increasingly being used to reduce or minimize the effects of confounding when estimating the effects of treatments, exposures, or interventions when using observational or non‐randomized data. Under the assumption of no unmeasured confounders, previous research has shown that propensity score methods allow for unbiased estimation of linear treatment effects (e.g., differences in means or proportions). However, in biomedical research, time‐to‐event outcomes occur frequently. There is a paucity of research into the performance of different propensity score methods for estimating the effect of treatment on time‐to‐event outcomes. Furthermore, propensity score methods allow for the estimation of marginal or population‐average treatment effects. We conducted an extensive series of Monte Carlo simulations to examine the performance of propensity score matching (1:1 greedy nearest‐neighbor matching within propensity score calipers), stratification on the propensity score, inverse probability of treatment weighting (IPTW) using the propensity score, and covariate adjustment using the propensity score to estimate marginal hazard ratios. We found that both propensity score matching and IPTW using the propensity score allow for the estimation of marginal hazard ratios with minimal bias. Of these two approaches, IPTW using the propensity score resulted in estimates with lower mean squared error when estimating the effect of treatment in the treated. Stratification on the propensity score and covariate adjustment using the propensity score result in biased estimation of both marginal and conditional hazard ratios. Applied researchers are encouraged to use propensity score matching and IPTW using the propensity score when estimating the relative effect of treatment on time‐to‐event outcomes. Copyright © 2012 John Wiley & Sons, Ltd.     

Assessing the performance of the generalized propensity score for estimating the effect of quantitative or continuous exposures on binary outcomes

Propensity score methods are increasingly being used to estimate the effects of treatments and exposures when using observational data. The propensity score was initially developed for use with binary exposures. The generalized propensity score (GPS) is an extension of the propensity score for use with quantitative or continuous exposures (eg, dose or quantity of medication, income, or years of education). We used Monte Carlo simulations to examine the performance of different methods of using the GPS to estimate the effect of continuous exposures on binary outcomes. We examined covariate adjustment using the GPS and weighting using weights based on the inverse of the GPS. We examined both the use of ordinary least squares to estimate the propensity function and the use of the covariate balancing propensity score algorithm. The use of methods based on the GPS was compared with the use of G‐computation. All methods resulted in essentially unbiased estimation of the population dose‐response function. However, GPS‐based weighting tended to result in estimates that displayed greater variability and had higher mean squared error when the magnitude of confounding was strong. Of the methods based on the GPS, covariate adjustment using the GPS tended to result in estimates with lower variability and mean squared error when the magnitude of confounding was strong. We illustrate the application of these methods by estimating the effect of average neighborhood income on the probability of death within 1 year of hospitalization for an acute myocardial infarction.     

Multiple imputation procedures for estimating causal effects with multiple treatments with application to the comparison of healthcare providers

Choosing between multiple healthcare providers requires us to simultaneously compare the expected outcomes under each provider. This comparison is complex because the composition of patients treated by each provider may differ. Similar issues arise when simultaneously comparing the adverse effects of interventions using non-randomized data. To simultaneously estimate the effects of multiple providers/interventions we propose procedures that explicitly impute the set of potential outcomes for each subject. The procedures are based on different specifications of the generalized additive models (GAM) and the Bayesian additive regression trees (BART). We compare the performance of the proposed procedures to previously proposed matching and weighting procedures using an extensive simulation study for continuous outcomes. Our simulations show that when the distributions of the covariates across treatment groups have adequate overlap, the multiple imputation procedures based on separate BART or GAM models in each treatment group are generally superior to weighting based methods and have similar and sometimes better performance than matching on the logit of the generalized propensity score. Another advantage of these multiple imputation procedures is the ability to provide point and interval estimates to a wide range of causal effect estimands. We apply the proposed procedures to comparing multiple nursing homes in Massachusetts for readmission outcomes. The proposed approach can be applied to other causal effects applications with multiple treatments.     

基于倾向值匹配的中医住院医师规范化培训现状研究

目的:中医住培是推动双规合一培养模式,提高中医医师培养质量的重要途径。通过比较中医住培与非中医住培现状,了解中医学员对住培工作的评价,为针对性地改善中医住培效果提供依据。方法:以云南省1502名住培学员为调查对象,采用倾向评分匹配法减少偏差和混杂变量的影响,比较不同类型学员对住培工作的评价。结果:中医学员在培训方法、组织管理、培训收入等方面满意度低于非中医学员,中医学员认为住培负责人重视度较低,中医学员培训后综合能力自评结果较差。经倾向评分法匹配中医与非中医学员基线资料后,回归结果显示,工资补贴、负责人重视程度、培训轮转安排和双向反馈制度对住培学员满意度有显著影响。结论:中医住培有着师徒传承、辨证论治的特点,目前仍处在效仿西医类培训的阶段,还需摸索出一套适合中医特色的培训政策制度。同时,中医学员认同感和归属感有待加强,未来仍需进一步完善中医住培配套管理制度,提高住院医师培训质量。     

The performance of different propensity score methods for estimating marginal odds ratios

The propensity score which is the probability of exposure to a specific treatment conditional on observed variables. Conditioning on the propensity score results in unbiased estimation of the expected difference in observed responses to two treatments. In the medical literature, propensity score methods are frequently used for estimating odds ratios. The performance of propensity score methods for estimating marginal odds ratios has not been studied. We performed a series of Monte Carlo simulations to assess the performance of propensity score matching, stratifying on the propensity score, and covariate adjustment using the propensity score to estimate marginal odds ratios. We assessed bias, precision, and mean-squared error (MSE) of the propensity score estimators, in addition to the proportion of bias eliminated due to conditioning on the propensity score. When the true marginal odds ratio was one, then matching on the propensity score and covariate adjustment using the propensity score resulted in unbiased estimation of the true treatment effect, whereas stratification on the propensity score resulted in minor bias in estimating the true marginal odds ratio. When the true marginal odds ratio ranged from 2 to 10, then matching on the propensity score resulted in the least bias, with a relative biases ranging from 2.3 to 13.3 per cent. Stratifying on the propensity score resulted in moderate bias, with relative biases ranging from 15.8 to 59.2 per cent. For both methods, relative bias was proportional to the true odds ratio. Finally, matching on the propensity score tended to result in estimators with the lowest MSE.     

The performance of different propensity-score methods for estimating relative risks

OBJECTIVES: The propensity score is the probability of treatment conditional on observed variables. Conditioning on the propensity-score results in unbiased estimation of the expected difference in observed responses to two treatments. The performance of propensity-score methods for estimating relative risks has not been studied. STUDY DESIGN AND SETTING: Monte Carlo simulations were used to assess the performance of matching, stratification, and covariate adjustment using the propensity score to estimate relative risks. RESULTS: Matching on the propensity score and stratification on the quintiles of the propensity score resulted in estimates of relative risk with similar mean squared error (MSE). Propensity-score matching resulted in estimates with less bias, whereas stratification on the propensity score resulted in estimates of with greater precision. Including only variables associated with the outcome or including only the true confounders in the propensity-score model resulted in estimates with lower MSE than did including all variables associated with treatment or all measured variables in the propensity-score model. CONCLUSIONS: When estimating relative risks, propensity-score matching resulted in estimates with less bias than did stratification on the quintiles of the propensity score, but stratification on the quintiles of the propensity score resulted in estimates with greater precision.     

A propensity score approach to estimating the cost-effectiveness of medical therapies from observational data

Health summary measures are commonly used by policy makers to help make decisions on the allocation of societal resources for competing medical treatments. The net monetary benefit is a health summary measure that overcomes the statistical limitations of a popular measure namely, the cost-effectiveness ratio. We introduce a linear model framework to estimate propensity score adjusted net monetary benefit. This method provides less biased estimates in the presence of significant differences in baseline measures and demographic characteristics between treatment groups in quasi-randomized or observational studies. Simulation studies were conducted to better understand the utility of propensity score adjusted estimates of net monetary benefits when important covariates are unobserved. The results indicated that the propensity score adjusted net monetary benefit provides a robust measure of cost-effectiveness in the presence of hidden bias. The methods are illustrated using data from SEER-Medicare for the treatment of bladder cancer.     

Generalizing Observational Study Results: Applying Propensity Score Methods to Complex Surveys

ObjectiveTo provide a tutorial for using propensity score methods with complex survey data.Data SourcesSimulated data and the 2008 Medical Expenditure Panel Survey.Study DesignUsing simulation, we compared the following methods for estimating the treatment effect: a naïve estimate (ignoring both survey weights and propensity scores), survey weighting, propensity score methods (nearest neighbor matching, weighting, and subclassification), and propensity score methods in combination with survey weighting. Methods are compared in terms of bias and 95 percent confidence interval coverage. In Example 2, we used these methods to estimate the effect on health care spending of having a generalist versus a specialist as a usual source of care.Principal FindingsIn general, combining a propensity score method and survey weighting is necessary to achieve unbiased treatment effect estimates that are generalizable to the original survey target population.ConclusionsPropensity score methods are an essential tool for addressing confounding in observational studies. Ignoring survey weights may lead to results that are not generalizable to the survey target population. This paper clarifies the appropriate inferences for different propensity score methods and suggests guidelines for selecting an appropriate propensity score method based on a researcher’s goal.     

Should a propensity score model be super? The utility of ensemble procedures for causal adjustment

In investigations of the effect of treatment on outcome, the propensity score is a tool to eliminate imbalance in the distribution of confounding variables between treatment groups. Recent work has suggested that Super Learner, an ensemble method, outperforms logistic regression in nonlinear settings; however, experience with real-data analyses tends to show overfitting of the propensity score model using this approach. We investigated a wide range of simulated settings of varying complexities including simulations based on real data to compare the performances of logistic regression, generalized boosted models, and Super Learner in providing balance and for estimating the average treatment effect via propensity score regression, propensity score matching, and inverse probability of treatment weighting. We found that Super Learner and logistic regression are comparable in terms of covariate balance, bias, and mean squared error (MSE); however, Super Learner is computationally very expensive thus leaving no clear advantage to the more complex approach. Propensity scores estimated by generalized boosted models were inferior to the other two estimation approaches. We also found that propensity score regression adjustment was superior to either matching or inverse weighting when the form of the dependence on the treatment on the outcome is correctly specified.     

Propensity score estimation with missing values using a multiple imputation missingness pattern (MIMP) approach

Propensity scores have been used widely as a bias reduction method to estimate the treatment effect in nonrandomized studies. Since many covariates are generally included in the model for estimating the propensity scores, the proportion of subjects with at least one missing covariate could be large. While many methods have been proposed for propensity score‐based estimation in the presence of missing covariates, little has been published comparing the performance of these methods. In this article we propose a novel method called multiple imputation missingness pattern (MIMP) and compare it with the naive estimator (ignoring propensity score) and three commonly used methods of handling missing covariates in propensity score‐based estimation (separate estimation of propensity scores within each pattern of missing data, multiple imputation and discarding missing data) under different mechanisms of missing data and degree of correlation among covariates. Simulation shows that all adjusted estimators are much less biased than the naive estimator. Under certain conditions MIMP provides benefits (smaller bias and mean‐squared error) compared with existing alternatives. Copyright © 2009 John Wiley & Sons, Ltd.     

不同混杂结构下广义倾向性评分法的模拟研究及应用

目的 通过构建存在不同混杂结构的广义倾向性评分(generalized propensity score,GPS)模型和结局模型,探索比较三种GPS估计法:广义倾向性评分-最小二乘法(generalized propensity score-ordinary least squares,GPS-OLS),广义倾向性评分...     

The use of bootstrapping when using propensity‐score matching without replacement: a simulation study

Propensity‐score matching is frequently used to estimate the effect of treatments, exposures, and interventions when using observational data. An important issue when using propensity‐score matching is how to estimate the standard error of the estimated treatment effect. Accurate variance estimation permits construction of confidence intervals that have the advertised coverage rates and tests of statistical significance that have the correct type I error rates. There is disagreement in the literature as to how standard errors should be estimated. The bootstrap is a commonly used resampling method that permits estimation of the sampling variability of estimated parameters. Bootstrap methods are rarely used in conjunction with propensity‐score matching. We propose two different bootstrap methods for use when using propensity‐score matching without replacementand examined their performance with a series of Monte Carlo simulations. The first method involved drawing bootstrap samples from the matched pairs in the propensity‐score‐matched sample. The second method involved drawing bootstrap samples from the original sample and estimating the propensity score separately in each bootstrap sample and creating a matched sample within each of these bootstrap samples. The former approach was found to result in estimates of the standard error that were closer to the empirical standard deviation of the sampling distribution of estimated effects. © 2014 The Authors. Statistics in Medicine Published by John Wiley & Sons, Ltd.