A comparison of propensity score methods: a case-study estimating the effectiveness of post-AMI statin use

There is an increasing interest in the use of propensity score methods to estimate causal effects in observational studies. However, recent systematic reviews have demonstrated that propensity score methods are inconsistently used and frequently poorly applied in the medical literature. In this study, we compared the following propensity score methods for estimating the reduction in all-cause mortality due to statin therapy for patients hospitalized with acute myocardial infarction: propensity-score matching, stratification using the propensity score, covariate adjustment using the propensity score, and weighting using the propensity score. We used propensity score methods to estimate both adjusted treated effects and the absolute and relative risk reduction in all-cause mortality. We also examined the use of statistical hypothesis testing, standardized differences, box plots, non-parametric density estimates, and quantile-quantile plots to assess residual confounding that remained after stratification or matching on the propensity score. Estimates of the absolute reduction in 3-year mortality ranged from 2.1 to 4.5 per cent, while estimates of the relative risk reduction ranged from 13.3 to 17.0 per cent. Adjusted estimates of the reduction in the odds of 3-year death varied from 15 to 24 per cent across the different propensity score methods.     

Costs and quality of hospitals in different health care systems: a multi‐level approach with propensity score matching

Cross‐country comparisons of costs and quality between hospitals are often made at the macro level. The goal of this study was to explore methods to compare micro‐level data from hospitals in different health care systems. To do so, we developed a multi‐level framework in combination with a propensity score matching technique using similarly structured data for patients receiving treatment for acute myocardial infarction in German and US Veterans Health Administration hospitals. Our case study shows important differences in results between multi‐level regressions based on matched and unmatched samples. We conclude that propensity score matching techniques are an appropriate way to deal with the usual baseline imbalances across the samples from different countries. Multi‐level models are recommendable to consider the clustered structure of the data when patient‐level data from different hospitals and health care systems are compared. The results provide an important justification for exploring new ways in performing health system comparisons. Copyright © 2010 John Wiley & Sons, Ltd.     

Interval estimation for treatment effects using propensity score matching

In causal studies without random assignment of treatment, causal effects can be estimated using matched treated and control samples, where matches are obtained using estimated propensity scores. Propensity score matching can reduce bias in treatment effect estimators in cases where the matched samples have overlapping covariate distributions. Despite its application in many applied problems, there is no universally employed approach to interval estimation when using propensity score matching. In this article, we present and evaluate approaches to interval estimation when using propensity score matching.     

On regression adjustment for the propensity score

Propensity scores are widely adopted in observational research because they enable adjustment for high‐dimensional confounders without requiring models for their association with the outcome of interest. The results of statistical analyses based on stratification, matching or inverse weighting by the propensity score are therefore less susceptible to model extrapolation than those based solely on outcome regression models. This is attractive because extrapolation in outcome regression models may be alarming, yet difficult to diagnose, when the exposed and unexposed individuals have very different covariate distributions. Standard regression adjustment for the propensity score forms an alternative to the aforementioned propensity score methods, but the benefits of this are less clear because it still involves modelling the outcome in addition to the propensity score. In this article, we develop novel insights into the properties of this adjustment method. We demonstrate that standard tests of the null hypothesis of no exposure effect (based on robust variance estimators), as well as particular standardised effects obtained from such adjusted regression models, are robust against misspecification of the outcome model when a propensity score model is correctly specified; they are thus not vulnerable to the aforementioned problem of extrapolation. We moreover propose efficient estimators for these standardised effects, which retain a useful causal interpretation even when the propensity score model is misspecified, provided the outcome regression model is correctly specified. Copyright © 2014 John Wiley & Sons, Ltd.     

The performance of different propensity score methods for estimating marginal odds ratios

The propensity score which is the probability of exposure to a specific treatment conditional on observed variables. Conditioning on the propensity score results in unbiased estimation of the expected difference in observed responses to two treatments. In the medical literature, propensity score methods are frequently used for estimating odds ratios. The performance of propensity score methods for estimating marginal odds ratios has not been studied. We performed a series of Monte Carlo simulations to assess the performance of propensity score matching, stratifying on the propensity score, and covariate adjustment using the propensity score to estimate marginal odds ratios. We assessed bias, precision, and mean-squared error (MSE) of the propensity score estimators, in addition to the proportion of bias eliminated due to conditioning on the propensity score. When the true marginal odds ratio was one, then matching on the propensity score and covariate adjustment using the propensity score resulted in unbiased estimation of the true treatment effect, whereas stratification on the propensity score resulted in minor bias in estimating the true marginal odds ratio. When the true marginal odds ratio ranged from 2 to 10, then matching on the propensity score resulted in the least bias, with a relative biases ranging from 2.3 to 13.3 per cent. Stratifying on the propensity score resulted in moderate bias, with relative biases ranging from 15.8 to 59.2 per cent. For both methods, relative bias was proportional to the true odds ratio. Finally, matching on the propensity score tended to result in estimators with the lowest MSE.     

基于倾向指数匹配法的肝癌患者生存分析

  目的  基于倾向性评分逆概率加权法（IPTW）评价手术、放疗和联合治疗3种方式治疗胃癌患者的疗效, 为胃癌的治疗提供参考依据。  方法  收集2004年1月 — 2013年12月美国国家癌症研究所监测、流行病学与预后项目（SEER）数据库中经胃镜病理诊断确诊的7 005例胃癌患者数据,其中接受手术治疗者3 983例、接受放疗者795例、接受联合治疗者2 227例;采用倾向性评分IPTW法以生存时间和结局为效应指标,分析不同的治疗方法对胃癌患者生存率的影响。  结果  手术组、放疗组和联合治疗组胃癌患者倾向性评分IPTW法加权前中位生存期分别为30、9和38个月,加权后分别为25、11和38个月,加权前、后3组胃癌患者生存曲线间差异均有统计学意义（均P < 0.001）,联合治疗组患者加权前、后的预后均优于手术组和放疗组患者;加权后大部分基线特征绝对标准化平均差异（ASMD）减小,且均 < 0.2,3组患者均衡效果较好;在控制了性别、年龄、种族、婚姻状况、肿瘤大小、病理分化、肿瘤分期、T分期、N分期和远处转移情况等混杂因素后,多因素Cox回归分析结果显示,与手术组胃癌患者相比,放疗组胃癌患者的预后较差（HR = 2.044,95 % CI = 1.770～2.361）,联合治疗组胃癌患者的预后较好（HR = 0.630,95 % CI = 0.573～0.694）。  结论  联合治疗方法疗效优于手术和放疗治疗方法。     

Assessing the performance of the generalized propensity score for estimating the effect of quantitative or continuous exposures on binary outcomes

Propensity score methods are increasingly being used to estimate the effects of treatments and exposures when using observational data. The propensity score was initially developed for use with binary exposures. The generalized propensity score (GPS) is an extension of the propensity score for use with quantitative or continuous exposures (eg, dose or quantity of medication, income, or years of education). We used Monte Carlo simulations to examine the performance of different methods of using the GPS to estimate the effect of continuous exposures on binary outcomes. We examined covariate adjustment using the GPS and weighting using weights based on the inverse of the GPS. We examined both the use of ordinary least squares to estimate the propensity function and the use of the covariate balancing propensity score algorithm. The use of methods based on the GPS was compared with the use of G‐computation. All methods resulted in essentially unbiased estimation of the population dose‐response function. However, GPS‐based weighting tended to result in estimates that displayed greater variability and had higher mean squared error when the magnitude of confounding was strong. Of the methods based on the GPS, covariate adjustment using the GPS tended to result in estimates with lower variability and mean squared error when the magnitude of confounding was strong. We illustrate the application of these methods by estimating the effect of average neighborhood income on the probability of death within 1 year of hospitalization for an acute myocardial infarction.     

The use of propensity score methods with survival or time‐to‐event outcomes: reporting measures of effect similar to those used in randomized experiments

Propensity score methods are increasingly being used to estimate causal treatment effects in observational studies. In medical and epidemiological studies, outcomes are frequently time‐to‐event in nature. Propensity‐score methods are often applied incorrectly when estimating the effect of treatment on time‐to‐event outcomes. This article describes how two different propensity score methods (matching and inverse probability of treatment weighting) can be used to estimate the measures of effect that are frequently reported in randomized controlled trials: (i) marginal survival curves, which describe survival in the population if all subjects were treated or if all subjects were untreated; and (ii) marginal hazard ratios. The use of these propensity score methods allows one to replicate the measures of effect that are commonly reported in randomized controlled trials with time‐to‐event outcomes: both absolute and relative reductions in the probability of an event occurring can be determined. We also provide guidance on variable selection for the propensity score model, highlight methods for assessing the balance of baseline covariates between treated and untreated subjects, and describe the implementation of a sensitivity analysis to assess the effect of unmeasured confounding variables on the estimated treatment effect when outcomes are time‐to‐event in nature. The methods in the paper are illustrated by estimating the effect of discharge statin prescribing on the risk of death in a sample of patients hospitalized with acute myocardial infarction. In this tutorial article, we describe and illustrate all the steps necessary to conduct a comprehensive analysis of the effect of treatment on time‐to‐event outcomes. © 2013 The authors. Statistics in Medicine published by John Wiley & Sons, Ltd.     

The performance of different propensity score methods for estimating marginal hazard ratios

Propensity score methods are increasingly being used to reduce or minimize the effects of confounding when estimating the effects of treatments, exposures, or interventions when using observational or non‐randomized data. Under the assumption of no unmeasured confounders, previous research has shown that propensity score methods allow for unbiased estimation of linear treatment effects (e.g., differences in means or proportions). However, in biomedical research, time‐to‐event outcomes occur frequently. There is a paucity of research into the performance of different propensity score methods for estimating the effect of treatment on time‐to‐event outcomes. Furthermore, propensity score methods allow for the estimation of marginal or population‐average treatment effects. We conducted an extensive series of Monte Carlo simulations to examine the performance of propensity score matching (1:1 greedy nearest‐neighbor matching within propensity score calipers), stratification on the propensity score, inverse probability of treatment weighting (IPTW) using the propensity score, and covariate adjustment using the propensity score to estimate marginal hazard ratios. We found that both propensity score matching and IPTW using the propensity score allow for the estimation of marginal hazard ratios with minimal bias. Of these two approaches, IPTW using the propensity score resulted in estimates with lower mean squared error when estimating the effect of treatment in the treated. Stratification on the propensity score and covariate adjustment using the propensity score result in biased estimation of both marginal and conditional hazard ratios. Applied researchers are encouraged to use propensity score matching and IPTW using the propensity score when estimating the relative effect of treatment on time‐to‐event outcomes. Copyright © 2012 John Wiley & Sons, Ltd.     

A comparison of 12 algorithms for matching on the propensity score

Propensity‐score matching is increasingly being used to reduce the confounding that can occur in observational studies examining the effects of treatments or interventions on outcomes. We used Monte Carlo simulations to examine the following algorithms for forming matched pairs of treated and untreated subjects: optimal matching, greedy nearest neighbor matching without replacement, and greedy nearest neighbor matching without replacement within specified caliper widths. For each of the latter two algorithms, we examined four different sub‐algorithms defined by the order in which treated subjects were selected for matching to an untreated subject: lowest to highest propensity score, highest to lowest propensity score, best match first, and random order. We also examined matching with replacement. We found that (i) nearest neighbor matching induced the same balance in baseline covariates as did optimal matching; (ii) when at least some of the covariates were continuous, caliper matching tended to induce balance on baseline covariates that was at least as good as the other algorithms; (iii) caliper matching tended to result in estimates of treatment effect with less bias compared with optimal and nearest neighbor matching; (iv) optimal and nearest neighbor matching resulted in estimates of treatment effect with negligibly less variability than did caliper matching; (v) caliper matching had amongst the best performance when assessed using mean squared error; (vi) the order in which treated subjects were selected for matching had at most a modest effect on estimation; and (vii) matching with replacement did not have superior performance compared with caliper matching without replacement. © 2013 The Authors. Statistics in Medicine published by John Wiley & Sons, Ltd.     

The added value of propensity score matching when using health‐related quality of life reference data

Direct comparisons of health‐related quality of life (HRQoL) outcomes between non‐randomized groups might be biased, as outcomes are confounded by imbalance in pre‐treatment patient characteristics. Such bias can be reduced by adjusting on observed covariates. This is the setting of HRQoL comparisons with reference data, where age and gender adjustment is commonly used for this purpose. However, other observed covariates can be used to lessen this bias and yield more precise estimates. The objective of this study is to show that more accurate HRQoL comparisons with reference data can be obtained, accounting for few covariates in addition to age and gender by a propensity score matching approach. Copyright © 2013 John Wiley & Sons, Ltd.     

Nonsampling measurement error in administrative data: Implications for economic evaluations

Administrative databases are increasingly being used to measure resource use in economic evaluations. Traditionally, it is assumed that any measurement error within the resource data source is stochastic and uncorrelated with group assignment. If the error is correlated with characteristics of the service delivery system and/or correlated by group, the estimated differences in consumption between groups may be reflecting errors in measurement rather than treatment effects. This paper is concerned with the effect of nonsampling measurement error on the internal and external validity of cost estimates based on data drawn from administrative record systems. Two service delivery characteristics, ownership form and financial incentives, are likely to influence systematically an agency's data collection effort. Using data from three community mental health centres located in three different states, evidence of systematic differences in data quality was found; private agencies with reimbursement property rights had higher quality data than public agencies without property rights. Simple tests for detecting variation in service use and costs data and cost-effective solutions for managing these problems are proposed.     

A propensity score approach to estimating the cost-effectiveness of medical therapies from observational data

Health summary measures are commonly used by policy makers to help make decisions on the allocation of societal resources for competing medical treatments. The net monetary benefit is a health summary measure that overcomes the statistical limitations of a popular measure namely, the cost-effectiveness ratio. We introduce a linear model framework to estimate propensity score adjusted net monetary benefit. This method provides less biased estimates in the presence of significant differences in baseline measures and demographic characteristics between treatment groups in quasi-randomized or observational studies. Simulation studies were conducted to better understand the utility of propensity score adjusted estimates of net monetary benefits when important covariates are unobserved. The results indicated that the propensity score adjusted net monetary benefit provides a robust measure of cost-effectiveness in the presence of hidden bias. The methods are illustrated using data from SEER-Medicare for the treatment of bladder cancer.     

Estimation of treatment effect adjusting for treatment changes using the intensity score method: application to a large primary prevention study for coronary events (MEGA study)

The MEGA study was a prospective, randomized, open-labeled, blinded-endpoints study conducted in Japan to evaluate the primary preventive effect of pravastatin against coronary heart disease (CHD), in which 8214 subjects were randomized to diet or diet plus pravastatin. The intention-to-treat (ITT) analysis showed that pravastatin reduced the incidence of CHD (hazard ratio=0.67; 95 per cent confidence interval (CI): 0.49-0.91) and of stroke events, which was the secondary endpoint in the MEGA study (hazard ratio=0.83; 95 per cent CI: 0.57-1.21). Owing to considerable treatment changes, it is also of interest to estimate the causal effect of treatment that would have been observed had all patients complied with the treatment to which they were assigned. In this paper, we present an intensity score method developed for clinical trials with time-to-event outcomes that correct for treatment changes during follow-up. The proposed method can be easily extended to the estimation of time-dependent treatment effects, where the technique of g-estimation has been difficult to apply in practice. We compared the performances of the proposed method with other methods (as-treated, ITT, and g-estimation analysis) through simulation studies, which showed that the intensity score estimator was unbiased and more efficient. Applying the proposed method to the MEGA study data, several prognostic factors were associated with the process of treatment changes, and after adjusting for these treatment changes, larger treatment effects for pravastatin were observed for both CHD and stroke events. The proposed method provides a valuable and flexible approach for estimating treatment effect adjusting for non-random non-compliance.     

Survival models for cost data: the forgotten additive approach

The usage of the Aalen additive approach is proposed to model cost data. Using a Monte Carlo simulation, in a wide set of scenarios, we showed that the Aalen model is performing well and can be a reasonable alternative to the standard Gamma regression models. In addition, with reference to the COSTAMI trial data, we highlighted the ability of the Aalen model to offer additional information about the relationships between costs and specific covariates, as compared with standard regression techniques.     

Conditioning on the propensity score can result in biased estimation of common measures of treatment effect: a Monte Carlo study

Propensity score methods are increasingly being used to estimate causal treatment effects in the medical literature. Conditioning on the propensity score results in unbiased estimation of the expected difference in observed responses to two treatments. The degree to which conditioning on the propensity score introduces bias into the estimation of the conditional odds ratio or conditional hazard ratio, which are frequently used as measures of treatment effect in observational studies, has not been extensively studied. We conducted Monte Carlo simulations to determine the degree to which propensity score matching, stratification on the quintiles of the propensity score, and covariate adjustment using the propensity score result in biased estimation of conditional odds ratios, hazard ratios, and rate ratios. We found that conditioning on the propensity score resulted in biased estimation of the true conditional odds ratio and the true conditional hazard ratio. In all scenarios examined, treatment effects were biased towards the null treatment effect. However, conditioning on the propensity score did not result in biased estimation of the true conditional rate ratio. In contrast, conventional regression methods allowed unbiased estimation of the true conditional treatment effect when all variables associated with the outcome were included in the regression model. The observed bias in propensity score methods is due to the fact that regression models allow one to estimate conditional treatment effects, whereas propensity score methods allow one to estimate marginal treatment effects. In several settings with non-linear treatment effects, marginal and conditional treatment effects do not coincide.     

基于倾向性评分匹配方法的孕前使用避孕药和妊娠天数的相关性分析

目的 探讨孕前使用避孕药对妊娠天数的影响。 方法 采用倾向性匹配评分方法,以基础资料为匹配条件,将使用避孕药者设为观察组,按照1∶4匹配未使用者为对照组。随访获取妊娠天数资料,进而探讨孕前使用避孕药和妊娠天数的关系。 结果 共匹配避孕药组241例,对照组964例,匹配以后各项资料均未见显著性差异(P＞0.05)。Cox回归分析发现,在调整基础资料后,使用避孕药可使妊娠天数缩短,危害比(hazard ratio,HR)为0.85(95%CI:0.74~0.98)。亚组分析发现,年龄≥31岁、体质指数≥24、无妊娠史三个亚组,孕前服用避孕药使妊娠天数缩短,HR(95%CI)分别为0.76(0.59,0.97)、0.67 (0.46,0.98)、0.78(0.63,0.96)。 结论 孕前服用避孕药可能缩短妊娠天数,特别是高龄、肥胖、初次妊娠女性,更应注意避孕药的使用。     

Propensity score matching with clustered data. An application to the estimation of the impact of caesarean section on the Apgar score

This article focuses on the implementation of propensity score matching for clustered data. Different approaches to reduce bias due to cluster‐level confounders are considered and compared using Monte Carlo simulations. We investigated methods that exploit the clustered structure of the data in two ways: in the estimation of the propensity score model (through the inclusion of fixed or random effects) or in the implementation of the matching algorithm. In addition to a pure within‐cluster matching, we also assessed the performance of a new approach, ‘preferential’ within‐cluster matching. This approach first searches for control units to be matched to treated units within the same cluster. If matching is not possible within‐cluster, then the algorithm searches in other clusters. All considered approaches successfully reduced the bias due to the omission of a cluster‐level confounder. The preferential within‐cluster matching approach, combining the advantages of within‐cluster and between‐cluster matching, showed a relatively good performance both in the presence of big and small clusters, and it was often the best method. An important advantage of this approach is that it reduces the number of unmatched units as compared with a pure within‐cluster matching. We applied these methods to the estimation of the effect of caesarean section on the Apgar score using birth register data. Copyright © 2016 John Wiley & Sons, Ltd.     

The use of quantile regression in health care research: a case study examining gender differences in the timeliness of thrombolytic therapy

Investigators are frequently interested in determining patient and system characteristics associated with delays in the provision of essential medical treatment. Investigators have typically used either multiple linear regression or Cox proportional hazards models to assess the impact of patient and system characteristics on the timeliness of medical treatment. A drawback to the use of these two methods is that they allow, at best, a partial exploration of how a distribution of delays in treatment or of waiting times changes with patient characteristics. In contrast, quantile regression models allow one to assess how any quantile of a conditional distribution changes with patient characteristics. We illustrate the utility of quantile regression by examining gender differences in the delivery of thrombolysis in patients with an acute myocardial infarction. We demonstrate that richer inferences can be drawn through the use of quantile regression. Females were more likely to experience delays in treatment compared to males. Furthermore, gender had a greater impact upon those patients who had the greatest delays in treatment. Investigators who want to determine how a distribution of delays in treatment or of waiting times changes with patient or system characteristics should consider complementing their analyses with the use of quantile regression.