On the use of propensity scores in principal causal effect estimation

We examine the practicality of propensity score methods for estimating causal treatment effects conditional on intermediate posttreatment outcomes (principal effects) in the context of randomized experiments. In particular, we focus on the sensitivity of principal causal effect estimates to violation of principal ignorability, which is the primary assumption that underlies the use of propensity score methods to estimate principal effects. Under principal ignorability (PI), principal strata membership is conditionally independent of the potential outcome under control given the pre‐treatment covariates; i.e. there are no differences in the potential outcomes under control across principal strata given the observed pretreatment covariates. Under this assumption, principal scores modeling principal strata membership can be estimated based solely on the observed covariates and used to predict strata membership and estimate principal effects. While this assumption underlies the use of propensity scores in this setting, sensitivity to violations of it has not been studied rigorously. In this paper, we explicitly define PI using the outcome model (although we do not actually use this outcome model in estimating principal scores) and systematically examine how deviations from the assumption affect estimates, including how the strength of association between principal stratum membership and covariates modifies the performance. We find that when PI is violated, very strong covariate predictors of stratum membership are needed to yield accurate estimates of principal effects. Copyright © 2009 John Wiley & Sons, Ltd.     

Sensitivity analysis for unmeasured confounding in principal stratification settings with binary variables

Within causal inference, principal stratification (PS) is a popular approach for dealing with intermediate variables, that is, variables affected by treatment that also potentially affect the response. However, when there exists unmeasured confounding in the treatment arms--as can happen in observational studies--causal estimands resulting from PS analyses can be biased. We identify the various pathways of confounding present in PS contexts and their effects for PS inference. We present model-based approaches for assessing the sensitivity of complier average causal effect estimates to unmeasured confounding in the setting of binary treatments, binary intermediate variables, and binary outcomes. These same approaches can be used to assess sensitivity to unknown direct effects of treatments on outcomes because, as we show, direct effects are operationally equivalent to one of the pathways of unmeasured confounding. We illustrate the methodology using a randomized study with artificially introduced confounding and a sensitivity analysis for an observational study of the effects of physical activity and body mass index on cardiovascular disease.     

Validation of surrogate endpoints in cancer clinical trials via principal stratification with an application to a prostate cancer trial

Increasing attention has been focused on the use and validation of surrogate endpoints in cancer clinical trials. Previous literature on validation of surrogate endpoints are classified into four approaches: the proportion explained approach; the indirect effects approach; the meta‐analytic approach; and the principal stratification approach. The mainstream in cancer research has seen the application of a meta‐analytic approach. However, VanderWeele (2013) showed that all four of these approaches potentially suffer from the surrogate paradox. It was also shown that, if a principal surrogate satisfies additional criteria called one‐sided average causal sufficiency, the surrogate cannot exhibit a surrogate paradox. Here, we propose a method for estimating principal effects under a monotonicity assumption. Specifically, we consider cancer clinical trials which compare a binary surrogate endpoint and a time‐to‐event clinical endpoint under two naturally ordered treatments (e.g. combined therapy vs. monotherapy). Estimation based on a mean score estimating equation will be implemented by the expectation‐maximization algorithm. We will also apply the proposed method as well as other surrogacy criteria to evaluate the surrogacy of prostate‐specific antigen using data from a phase III advanced prostate cancer trial, clarifying the complementary roles of both the principal stratification and meta‐analytic approaches in the evaluation of surrogate endpoints in cancer. Copyright © 2017 John Wiley & Sons, Ltd.     

Principal stratification for quantile causal effects under partial compliance

Within the principal stratification framework in causal inference, the majority of the literature has focused on binary compliance with an intervention and modelling means. Yet in some research areas, compliance is partial, and research questions—and hence analyses—are concerned with causal effects on (possibly high) quantiles rather than on shifts in average outcomes. Modelling partial compliance is challenging because it can suffer from lack of identifiability. We develop an approach to estimate quantile causal effects within a principal stratification framework, where principal strata are defined by the bivariate vector of (partial) compliance to the two levels of a binary intervention. We propose a conditional copula approach to impute the missing potential compliance and estimate the principal quantile treatment effect surface at high quantiles, allowing the copula association parameter to vary with the covariates. A bootstrap procedure is used to estimate the parameter to account for inflation due to imputation of missing compliance. Moreover, we describe precise assumptions on which the proposed approach is based, and investigate the finite sample behavior of our method by a simulation study. The proposed approach is used to study the 90th principal quantile treatment effect of executive stay-at-home orders on mitigating the risk of COVID-19 transmission in the United States.     

Mediation analysis with principal stratification

In assessing the mechanism of treatment efficacy in randomized clinical trials, investigators often perform mediation analyses by analyzing if the significant intent‐to‐treat treatment effect on outcome occurs through or around a third intermediate or mediating variable: indirect and direct effects, respectively. Standard mediation analyses assume sequential ignorability, i.e. conditional on covariates the intermediate or mediating factor is randomly assigned, as is the treatment in a randomized clinical trial. This research focuses on the application of the principal stratification (PS) approach for estimating the direct effect of a randomized treatment but without the standard sequential ignorability assumption. This approach is used to estimate the direct effect of treatment as a difference between expectations of potential outcomes within latent subgroups of participants for whom the intermediate variable behavior would be constant, regardless of the randomized treatment assignment. Using a Bayesian estimation procedure, we also assess the sensitivity of results based on the PS approach to heterogeneity of the variances among these principal strata. We assess this approach with simulations and apply it to two psychiatric examples. Both examples and the simulations indicated robustness of our findings to the homogeneous variance assumption. However, simulations showed that the magnitude of treatment effects derived under the PS approach were sensitive to model mis‐specification. Copyright © 2009 John Wiley & Sons, Ltd.     

The performance of different propensity‐score methods for estimating differences in proportions (risk differences or absolute risk reductions) in observational studies

Propensity score methods are increasingly being used to estimate the effects of treatments on health outcomes using observational data. There are four methods for using the propensity score to estimate treatment effects: covariate adjustment using the propensity score, stratification on the propensity score, propensity‐score matching, and inverse probability of treatment weighting (IPTW) using the propensity score. When outcomes are binary, the effect of treatment on the outcome can be described using odds ratios, relative risks, risk differences, or the number needed to treat. Several clinical commentators suggested that risk differences and numbers needed to treat are more meaningful for clinical decision making than are odds ratios or relative risks. However, there is a paucity of information about the relative performance of the different propensity‐score methods for estimating risk differences. We conducted a series of Monte Carlo simulations to examine this issue. We examined bias, variance estimation, coverage of confidence intervals, mean‐squared error (MSE), and type I error rates. A doubly robust version of IPTW had superior performance compared with the other propensity‐score methods. It resulted in unbiased estimation of risk differences, treatment effects with the lowest standard errors, confidence intervals with the correct coverage rates, and correct type I error rates. Stratification, matching on the propensity score, and covariate adjustment using the propensity score resulted in minor to modest bias in estimating risk differences. Estimators based on IPTW had lower MSE compared with other propensity‐score methods. Differences between IPTW and propensity‐score matching may reflect that these two methods estimate the average treatment effect and the average treatment effect for the treated, respectively. Copyright © 2010 John Wiley & Sons, Ltd.     

Joint modeling compliance and outcome for causal analysis in longitudinal studies

This article discusses joint modeling of compliance and outcome for longitudinal studies when noncompliance is present. We focus on two‐arm randomized longitudinal studies in which subjects are randomized at baseline, treatment is applied repeatedly over time, and compliance behaviors and clinical outcomes are measured and recorded repeatedly over time. In the proposed Markov compliance and outcome model, we use the potential outcome framework to define pre‐randomization principal strata from the joint distribution of compliance under treatment and control arms, and estimate the effect of treatment within each principal strata. Besides the causal effect of the treatment, our proposed model can estimate the impact of the causal effect of the treatment at a given time on future compliance. Bayesian methods are used to estimate the parameters. The results are illustrated using a study assessing the effect of cognitive behavior therapy on depression. A simulation study is used to assess the repeated sampling properties of the proposed model. Copyright © 2013 John Wiley & Sons, Ltd.     

Applying optimal model selection in principal stratification for causal inference

Noncompliance to treatment allocation is a key source of complication for causal inference. Efficacy estimation is likely to be compounded by the presence of noncompliance in both treatment arms of clinical trials where the intention‐to‐treat estimate provides a biased estimator for the true causal estimate even under homogeneous treatment effects assumption. Principal stratification method has been developed to address such posttreatment complications. The present work extends a principal stratification method that adjusts for noncompliance in two‐treatment arms trials by developing model selection for covariates predicting compliance to treatment in each arm. We apply the method to analyse data from the Esprit study, which was conducted to ascertain whether unopposed oestrogen (hormone replacement therapy) reduced the risk of further cardiac events in postmenopausal women who survive a first myocardial infarction. We adjust for noncompliance in both treatment arms under a Bayesian framework to produce causal risk ratio estimates for each principal stratum. For mild values of a sensitivity parameter and using separate predictors of compliance in each arm, principal stratification results suggested that compliance with hormone replacement therapy only would reduce the risk for death and myocardial reinfarction by about 47% and 25%, respectively, whereas compliance with either treatment would reduce the risk for death by 13% and reinfarction by 60% among the most compliant. However, the results were sensitive to the user‐defined sensitivity parameter. Copyright © 2012 John Wiley & Sons, Ltd.     

A Bayesian approach to estimating causal vaccine effects on binary post‐infection outcomes

To estimate causal effects of vaccine on post‐infection outcomes, Hudgens and Halloran (2006) defined a post‐infection causal vaccine efficacy estimand VE_I based on the principal stratification framework. They also derived closed forms for the maximum likelihood estimators of the causal estimand under some assumptions. Extending their research, we propose a Bayesian approach to estimating the causal vaccine effects on binary post‐infection outcomes. The identifiability of the causal vaccine effect VE_I is discussed under different assumptions on selection bias. The performance of the proposed Bayesian method is compared with the maximum likelihood method through simulation studies and two case studies — a clinical trial of a rotavirus vaccine candidate and a field study of pertussis vaccination. For both case studies, the Bayesian approach provided similar inference as the frequentist analysis. However, simulation studies with small sample sizes suggest that the Bayesian approach provides smaller bias and shorter confidence interval length. Copyright © 2015 John Wiley & Sons, Ltd.     

Bayesian sequential monitoring design for two‐arm randomized clinical trials with noncompliance

In early‐phase clinical trials, interim monitoring is commonly conducted based on the estimated intent‐to‐treat effect, which is subject to bias in the presence of noncompliance. To address this issue, we propose a Bayesian sequential monitoring trial design based on the estimation of the causal effect using a principal stratification approach. The proposed design simultaneously considers efficacy and toxicity outcomes and utilizes covariates to predict a patient's potential compliance behavior and identify the causal effects. Based on accumulating data, we continuously update the posterior estimates of the causal treatment effects and adaptively make the go/no‐go decision for the trial. Numerical results show that the proposed method has desirable operating characteristics and addresses the issue of noncompliance. Copyright © 2015 John Wiley & Sons, Ltd.     

Improved estimation of controlled direct effects in the presence of unmeasured confounding of intermediate variables

Adjusting for a causal intermediate is a common analytic strategy for estimating an average causal direct effect (ACDE). The ACDE is the component of the total exposure effect that is not relayed through the specified intermediate. Even if the total effect is unconfounded, the usual ACDE estimate may be biased when an unmeasured variable affects the intermediate and outcome variables. Using linear programming optimization to compute non-parametric bounds, we develop new ACDE estimators for binary measured variables in this causal structure, and use root mean square confounding bias (RMSB) to compare their performance with the usual stratified estimator in simulated distributions of target populations comprised of the 64 possible potential response types as well as distributions of target populations restricted to subsets of 18 or 12 potential response types defined by monotonicity or no-interactions assumptions of unit-level causal effects. We also consider target population distributions conditioned on fixed outcome risk among the unexposed, or fixed true ACDE in one stratum of the intermediate. Results show that a midpoint estimator constructed from the optimization bounds has consistently lower RMSB than the usual stratified estimator both unconditionally and conditioned on any risk in the unexposed. When conditioning on true ACDE, this midpoint estimator performs more poorly only when conditioned on an extreme true ACDE in one stratum of the intermediate, yet outperforms the stratified estimator in the other stratum when interaction is permitted. An alternate 'limit-modified crude' estimator can never perform less favourably than the stratified estimator, and often has lower RMSB.     

Sharp nonparametric bounds and randomization inference for treatment effects on an ordinal outcome

In clinical research, investigators are interested in inferring the average causal effect of a treatment. However, the causal parameter that can be used to derive the average causal effect is not well defined for ordinal outcomes. Although some definitions have been proposed, they are limited in that they are not identical to the well‐defined causal risk for a binary outcome, which is the simplest ordinal outcome. In this paper, we propose the use of a causal parameter for an ordinal outcome, defined as the proportion that a potential outcome under one treatment condition would not be smaller than that under the other condition. For a binary outcome, this proportion is identical to the causal risk. Unfortunately, the proposed causal parameter cannot be identified, even under randomization. Therefore, we present a numerical method to calculate the sharp nonparametric bounds within a sample, reflecting the impact of confounding. When the assumption of independent potential outcomes is included, the causal parameter can be identified when randomization is in play. Then, we present exact tests and the associated confidence intervals for the relative treatment effect using the randomization‐based approach, which are an extension of the existing methods for a binary outcome. Our methodologies are illustrated using data from an emetic prevention clinical trial.     

Using structural‐nested models to estimate the effect of cluster‐level adherence on individual‐level outcomes with a three‐armed cluster‐randomized trial

Much attention has been paid to estimating the causal effect of adherence to a randomized protocol using instrumental variables to adjust for unmeasured confounding. Researchers tend to use the instrumental variable within one of the three main frameworks: regression with an endogenous variable, principal stratification, or structural‐nested modeling. We found in our literature review that even in simple settings, causal interpretations of analyses with endogenous regressors can be ambiguous or rely on a strong assumption that can be difficult to interpret. Principal stratification and structural‐nested modeling are alternative frameworks that render unambiguous causal interpretations based on assumptions that are, arguably, easier to interpret. Our interest stems from a wish to estimate the effect of cluster‐level adherence on individual‐level binary outcomes with a three‐armed cluster‐randomized trial and polytomous adherence. Principal stratification approaches to this problem are quite challenging because of the sheer number of principal strata involved. Therefore, we developed a structural‐nested modeling approach and, in the process, extended the methodology to accommodate cluster‐randomized trials with unequal probability of selecting individuals. Furthermore, we developed a method to implement the approach with relatively simple programming. The approach works quite well, but when the structural‐nested model does not fit the data, there is no solution to the estimating equation. We investigate the performance of the approach using simulated data, and we also use the approach to estimate the effect on pupil absence of school‐level adherence to a randomized water, sanitation, and hygiene intervention in western Kenya. Copyright © 2013 John Wiley & Sons, Ltd.     

Comparing methods for estimation of heterogeneous treatment effects using observational data from health care databases

There is growing interest in using routinely collected data from health care databases to study the safety and effectiveness of therapies in “real‐world” conditions, as it can provide complementary evidence to that of randomized controlled trials. Causal inference from health care databases is challenging because the data are typically noisy, high dimensional, and most importantly, observational. It requires methods that can estimate heterogeneous treatment effects while controlling for confounding in high dimensions. Bayesian additive regression trees, causal forests, causal boosting, and causal multivariate adaptive regression splines are off‐the‐shelf methods that have shown good performance for estimation of heterogeneous treatment effects in observational studies of continuous outcomes. However, it is not clear how these methods would perform in health care database studies where outcomes are often binary and rare and data structures are complex. In this study, we evaluate these methods in simulation studies that recapitulate key characteristics of comparative effectiveness studies. We focus on the conditional average effect of a binary treatment on a binary outcome using the conditional risk difference as an estimand. To emulate health care database studies, we propose a simulation design where real covariate and treatment assignment data are used and only outcomes are simulated based on nonparametric models of the real outcomes. We apply this design to 4 published observational studies that used records from 2 major health care databases in the United States. Our results suggest that Bayesian additive regression trees and causal boosting consistently provide low bias in conditional risk difference estimates in the context of health care database studies.     

Causal inference in survival analysis using pseudo‐observations

Causal inference for non‐censored response variables, such as binary or quantitative outcomes, is often based on either (1) direct standardization (‘G‐formula’) or (2) inverse probability of treatment assignment weights (‘propensity score’). To do causal inference in survival analysis, one needs to address right‐censoring, and often, special techniques are required for that purpose. We will show how censoring can be dealt with ‘once and for all’ by means of so‐called pseudo‐observations when doing causal inference in survival analysis. The pseudo‐observations can be used as a replacement of the outcomes without censoring when applying ‘standard’ causal inference methods, such as (1) or (2) earlier. We study this idea for estimating the average causal effect of a binary treatment on the survival probability, the restricted mean lifetime, and the cumulative incidence in a competing risks situation. The methods will be illustrated in a small simulation study and via a study of patients with acute myeloid leukemia who received either myeloablative or non‐myeloablative conditioning before allogeneic hematopoetic cell transplantation. We will estimate the average causal effect of the conditioning regime on outcomes such as the 3‐year overall survival probability and the 3‐year risk of chronic graft‐versus‐host disease. Copyright © 2017 John Wiley & Sons, Ltd.     

The performance of different propensity score methods for estimating marginal hazard ratios

Propensity score methods are increasingly being used to reduce or minimize the effects of confounding when estimating the effects of treatments, exposures, or interventions when using observational or non‐randomized data. Under the assumption of no unmeasured confounders, previous research has shown that propensity score methods allow for unbiased estimation of linear treatment effects (e.g., differences in means or proportions). However, in biomedical research, time‐to‐event outcomes occur frequently. There is a paucity of research into the performance of different propensity score methods for estimating the effect of treatment on time‐to‐event outcomes. Furthermore, propensity score methods allow for the estimation of marginal or population‐average treatment effects. We conducted an extensive series of Monte Carlo simulations to examine the performance of propensity score matching (1:1 greedy nearest‐neighbor matching within propensity score calipers), stratification on the propensity score, inverse probability of treatment weighting (IPTW) using the propensity score, and covariate adjustment using the propensity score to estimate marginal hazard ratios. We found that both propensity score matching and IPTW using the propensity score allow for the estimation of marginal hazard ratios with minimal bias. Of these two approaches, IPTW using the propensity score resulted in estimates with lower mean squared error when estimating the effect of treatment in the treated. Stratification on the propensity score and covariate adjustment using the propensity score result in biased estimation of both marginal and conditional hazard ratios. Applied researchers are encouraged to use propensity score matching and IPTW using the propensity score when estimating the relative effect of treatment on time‐to‐event outcomes. Copyright © 2012 John Wiley & Sons, Ltd.     

Discovering subpopulation structure with latent class mixed models

The linear mixed model is a well-known method for incorporating heterogeneity (for example, subject-to-subject variation) into a statistical analysis for continuous responses. However heterogeneity cannot always be fully captured by the usual assumptions of normally distributed random effects. Latent class mixed models offer a way of incorporating additional heterogeneity which can be used to uncover distinct subpopulations, to incorporate correlated non-normally distributed outcomes and to classify individuals. The methodology is motivated with examples in health care studies and a detailed illustration is drawn from the Nutritional Prevention of Cancer trials. Latent class models are used with longitudinal data on prostate specific antigen (PSA) as well as incidence of prostate cancer. The models are extended to accommodate prostate cancer as a survival endpoint; this is compared to treating it as a binary endpoint. Four subpopulations are identified which differ both with regard to their PSA trajectories and their incidence rates of prostate cancer.     

Marginalized models for longitudinal ordinal data with application to quality of life studies

Random effects are often used in generalized linear models to explain the serial dependence for longitudinal categorical data. Marginalized random effects models (MREMs) for the analysis of longitudinal binary data have been proposed to permit likelihood-based estimation of marginal regression parameters. In this paper, we propose a model to extend the MREM to accommodate longitudinal ordinal data. Maximum marginal likelihood estimation is proposed utilizing quasi-Newton algorithms with Monte Carlo integration of the random effects. Our approach is applied to analyze the quality of life data from a recent colorectal cancer clinical trial. Dropout occurs at a high rate and is often due to tumor progression or death. To deal with events due to progression/death, we used a mixture model for the joint distribution of longitudinal measures and progression/death times and use principal stratification to draw causal inferences about survivors.     

Estimating drug effects in the presence of placebo response: Causal inference using growth mixture modeling

Placebo‐controlled randomized trials for antidepressants and other drugs often show a response for a sizeable percentage of the subjects in the placebo group. Potential placebo responders can be assumed to exist also in the drug treatment group, making it difficult to assess the drug effect. A key drug research focus should be to estimate the percentage of individuals among those who responded to the drug who would not have responded to the placebo (‘Drug Only Responders’). This paper investigates a finite mixture model approach to uncover percentages of up to four potential mixture components: Never Responders, Drug Only Responders, Placebo Only Responders, and Always Responders. Two examples are used to illustrate the modeling, a 12‐week antidepressant trial with a continuous outcome (Hamilton D score) and a 7‐week schizophrenia trial with a binary outcome (illness level). The approach is formulated in causal modeling terms using potential outcomes and principal stratification. Growth mixture modeling (GMM) with maximum‐likelihood estimation is used to uncover the different mixture components. The results point to the limitations of the conventional approach of comparing marginal response rates for drug and placebo groups. It is useful to augment such reporting with the GMM‐estimated prevalences for the four classes of subjects and the Drug Only Responder drug effect estimate. Copyright © 2009 John Wiley & Sons, Ltd.     

Bayesian inference for a principal stratum estimand to assess the treatment effect in a subgroup characterized by postrandomization event occurrence

The treatment effect in subgroups of patients is often of interest in randomized controlled clinical trials, as this may provide useful information on how to treat which patients best. When a specific subgroup is characterized by the absence of certain events that happen postrandomization, a naive analysis on the subset of patients without these events may be misleading. The principal stratification framework allows one to define an appropriate causal estimand in such settings. Statistical inference for the principal stratum estimand hinges on scientifically justified assumptions, which can be included with Bayesian methods through prior distributions. Our motivating example is a large randomized placebo-controlled trial of siponimod in patients with secondary progressive multiple sclerosis. The primary objective of this trial was to demonstrate the efficacy of siponimod relative to placebo in delaying disability progression for the whole study population. However, the treatment effect in the subgroup of patients who would not relapse during the trial is relevant from both a scientific and patient perspective. Assessing this subgroup treatment effect is challenging as there is strong evidence that siponimod reduces relapses. We describe in detail the scientific question of interest, the principal stratum estimand, the corresponding analysis method for binary endpoints, and sensitivity analyses. Although our work is motivated by a randomized clinical trial, the approach has broader appeal and could be adapted for observational studies.