Adaptive pre‐specification in randomized trials with and without pair‐matching

In randomized trials, adjustment for measured covariates during the analysis can reduce variance and increase power. To avoid misleading inference, the analysis plan must be pre‐specified. However, it is often unclear a priori which baseline covariates (if any) should be adjusted for in the analysis. Consider, for example, the Sustainable East Africa Research in Community Health (SEARCH) trial for HIV prevention and treatment. There are 16 matched pairs of communities and many potential adjustment variables, including region, HIV prevalence, male circumcision coverage, and measures of community‐level viral load. In this paper, we propose a rigorous procedure to data‐adaptively select the adjustment set, which maximizes the efficiency of the analysis. Specifically, we use cross‐validation to select from a pre‐specified library the candidate targeted maximum likelihood estimator (TMLE) that minimizes the estimated variance. For further gains in precision, we also propose a collaborative procedure for estimating the known exposure mechanism. Our small sample simulations demonstrate the promise of the methodology to maximize study power, while maintaining nominal confidence interval coverage. We show how our procedure can be tailored to the scientific question (intervention effect for the study sample vs. for the target population) and study design (pair‐matched or not). Copyright © 2016 John Wiley & Sons, Ltd.     

Targeted estimation and inference for the sample average treatment effect in trials with and without pair‐matching

In cluster randomized trials, the study units usually are not a simple random sample from some clearly defined target population. Instead, the target population tends to be hypothetical or ill‐defined, and the selection of study units tends to be systematic, driven by logistical and practical considerations. As a result, the population average treatment effect (PATE) may be neither well defined nor easily interpretable. In contrast, the sample average treatment effect (SATE) is the mean difference in the counterfactual outcomes for the study units. The sample parameter is easily interpretable and arguably the most relevant when the study units are not sampled from some specific super‐population of interest. Furthermore, in most settings, the sample parameter will be estimated more efficiently than the population parameter. To the best of our knowledge, this is the first paper to propose using targeted maximum likelihood estimation (TMLE) for estimation and inference of the sample effect in trials with and without pair‐matching. We study the asymptotic and finite sample properties of the TMLE for the sample effect and provide a conservative variance estimator. Finite sample simulations illustrate the potential gains in precision and power from selecting the sample effect as the target of inference. This work is motivated by the Sustainable East Africa Research in Community Health (SEARCH) study, a pair‐matched, community randomized trial to estimate the effect of population‐based HIV testing and streamlined ART on the 5‐year cumulative HIV incidence (NCT01864603). The proposed methodology will be used in the primary analysis for the SEARCH trial. Copyright © 2016 John Wiley & Sons, Ltd.     

A comparison of 12 algorithms for matching on the propensity score

Propensity‐score matching is increasingly being used to reduce the confounding that can occur in observational studies examining the effects of treatments or interventions on outcomes. We used Monte Carlo simulations to examine the following algorithms for forming matched pairs of treated and untreated subjects: optimal matching, greedy nearest neighbor matching without replacement, and greedy nearest neighbor matching without replacement within specified caliper widths. For each of the latter two algorithms, we examined four different sub‐algorithms defined by the order in which treated subjects were selected for matching to an untreated subject: lowest to highest propensity score, highest to lowest propensity score, best match first, and random order. We also examined matching with replacement. We found that (i) nearest neighbor matching induced the same balance in baseline covariates as did optimal matching; (ii) when at least some of the covariates were continuous, caliper matching tended to induce balance on baseline covariates that was at least as good as the other algorithms; (iii) caliper matching tended to result in estimates of treatment effect with less bias compared with optimal and nearest neighbor matching; (iv) optimal and nearest neighbor matching resulted in estimates of treatment effect with negligibly less variability than did caliper matching; (v) caliper matching had amongst the best performance when assessed using mean squared error; (vi) the order in which treated subjects were selected for matching had at most a modest effect on estimation; and (vii) matching with replacement did not have superior performance compared with caliper matching without replacement. © 2013 The Authors. Statistics in Medicine published by John Wiley & Sons, Ltd.     

A sensitivity analysis for missing outcomes due to truncation by death under the matched‐pairs design

The matched‐pairs design enables researchers to efficiently infer causal effects from randomized experiments. In this paper, we exploit the key feature of the matched‐pairs design and develop a sensitivity analysis for missing outcomes due to truncation by death, in which the outcomes of interest (e.g., quality of life measures) are not even well defined for some units (e.g., deceased patients). Our key idea is that if 2 nearly identical observations are paired prior to the randomization of the treatment, the missingness of one unit's outcome is informative about the potential missingness of the other unit's outcome under an alternative treatment condition. We consider the average treatment effect among always‐observed pairs (ATOP) whose units exhibit no missing outcome regardless of their treatment status. The naive estimator based on available pairs is unbiased for the ATOP if 2 units of the same pair are identical in terms of their missingness patterns. The proposed sensitivity analysis characterizes how the bounds of the ATOP widen as the degree of the within‐pair similarity decreases. We further extend the methodology to the matched‐pairs design in observational studies. Our simulation studies show that informative bounds can be obtained under some scenarios when the proportion of missing data is not too large. The proposed methodology is also applied to the randomized evaluation of the Mexican universal health insurance program. An open‐source software package is available for implementing the proposed research.     

Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity‐score matched samples

The propensity score is a subject's probability of treatment, conditional on observed baseline covariates. Conditional on the true propensity score, treated and untreated subjects have similar distributions of observed baseline covariates. Propensity‐score matching is a popular method of using the propensity score in the medical literature. Using this approach, matched sets of treated and untreated subjects with similar values of the propensity score are formed. Inferences about treatment effect made using propensity‐score matching are valid only if, in the matched sample, treated and untreated subjects have similar distributions of measured baseline covariates. In this paper we discuss the following methods for assessing whether the propensity score model has been correctly specified: comparing means and prevalences of baseline characteristics using standardized differences; ratios comparing the variance of continuous covariates between treated and untreated subjects; comparison of higher order moments and interactions; five‐number summaries; and graphical methods such as quantile–quantile plots, side‐by‐side boxplots, and non‐parametric density plots for comparing the distribution of baseline covariates between treatment groups. We describe methods to determine the sampling distribution of the standardized difference when the true standardized difference is equal to zero, thereby allowing one to determine the range of standardized differences that are plausible with the propensity score model having been correctly specified. We highlight the limitations of some previously used methods for assessing the adequacy of the specification of the propensity‐score model. In particular, methods based on comparing the distribution of the estimated propensity score between treated and untreated subjects are uninformative. Copyright © 2009 John Wiley & Sons, Ltd.     

Assessing the gain in efficiency due to matching in a community intervention study

COMMIT (Community Intervention Trial for Smoking Cessation) is a randomized study employing a matched pairs design. Pairs of communities were selected on the basis of their geographical proximity and were chosen to be matched on variables strongly expected to relate to the outcome variable, the smoking quit rate. However, quantitative information was not available to evaluate the efficiency gain from matching. We have used baseline smoking quit rates in the communities as a surrogate for the outcome measure to evaluate the gain in efficiency from the matching. Our method takes account of the possible imperfection of the surrogate as a representative of the true outcome. The method estimates an efficiency gain of at least 50 per cent using the matched design. We also evaluate the further gains in efficiency which would be made by using the baseline quit rate to balance the randomization.     

Using the entire history in the analysis of nested case cohort samples

Countermatching designs can provide more efficient estimates than simple matching or case–cohort designs in certain situations such as when good surrogate variables for an exposure of interest are available. We extend pseudolikelihood estimation for the Cox model under countermatching designs to models where time‐varying covariates are considered. We also implement pseudolikelihood with calibrated weights to improve efficiency in nested case–control designs in the presence of time‐varying variables. A simulation study is carried out, which considers four different scenarios including a binary time‐dependent variable, a continuous time‐dependent variable, and the case including interactions in each. Simulation results show that pseudolikelihood with calibrated weights under countermatching offers large gains in efficiency if compared to case–cohort. Pseudolikelihood with calibrated weights yielded more efficient estimators than pseudolikelihood estimators. Additionally, estimators were more efficient under countermatching than under case–cohort for the situations considered. The methods are illustrated using the Colorado Plateau uranium miners cohort. Furthermore, we present a general method to generate survival times with time‐varying covariates. Copyright © 2016 John Wiley & Sons, Ltd.     

Semiparametric time varying coefficient model for matched case‐crossover studies

In matched case‐crossover studies, it is generally accepted that the covariates on which a case and associated controls are matched cannot exert a confounding effect on independent predictors included in the conditional logistic regression model. This is because any stratum effect is removed by the conditioning on the fixed number of sets of the case and controls in the stratum. Hence, the conditional logistic regression model is not able to detect any effects associated with the matching covariates by stratum. However, some matching covariates such as time often play an important role as an effect modification leading to incorrect statistical estimation and prediction. Therefore, we propose three approaches to evaluate effect modification by time. The first is a parametric approach, the second is a semiparametric penalized approach, and the third is a semiparametric Bayesian approach. Our parametric approach is a two‐stage method, which uses conditional logistic regression in the first stage and then estimates polynomial regression in the second stage. Our semiparametric penalized and Bayesian approaches are one‐stage approaches developed by using regression splines. Our semiparametric one stage approach allows us to not only detect the parametric relationship between the predictor and binary outcomes, but also evaluate nonparametric relationships between the predictor and time. We demonstrate the advantage of our semiparametric one‐stage approaches using both a simulation study and an epidemiological example of a 1‐4 bi‐directional case‐crossover study of childhood aseptic meningitis with drinking water turbidity. We also provide statistical inference for the semiparametric Bayesian approach using Bayes Factors. Copyright © 2016 John Wiley & Sons, Ltd.     

Doubly robust inference for targeted minimum loss–based estimation in randomized trials with missing outcome data

Missing outcome data is a crucial threat to the validity of treatment effect estimates from randomized trials. The outcome distributions of participants with missing and observed data are often different, which increases bias. Causal inference methods may aid in reducing the bias and improving efficiency by incorporating baseline variables into the analysis. In particular, doubly robust estimators incorporate 2 nuisance parameters: the outcome regression and the missingness mechanism (ie, the probability of missingness conditional on treatment assignment and baseline variables), to adjust for differences in the observed and unobserved groups that can be explained by observed covariates. To consistently estimate the treatment effect, one of these nuisance parameters must be consistently estimated. Traditionally, nuisance parameters are estimated using parametric models, which often precludes consistency, particularly in moderate to high dimensions. Recent research on missing data has focused on data‐adaptive estimation to help achieve consistency, but the large sample properties of such methods are poorly understood. In this article, we discuss a doubly robust estimator that is consistent and asymptotically normal under data‐adaptive estimation of the nuisance parameters. We provide a formula for an asymptotically exact confidence interval under minimal assumptions. We show that our proposed estimator has smaller finite‐sample bias compared to standard doubly robust estimators. We present a simulation study demonstrating the enhanced performance of our estimators in terms of bias, efficiency, and coverage of the confidence intervals. We present the results of an illustrative example: a randomized, double‐blind phase 2/3 trial of antiretroviral therapy in HIV‐infected persons.     

Genotype‐based matching to correct for population stratification in large‐scale case‐control genetic association studies

Genome‐wide association studies are helping to dissect the etiology of complex diseases. Although case‐control association tests are generally more powerful than family‐based association tests, population stratification can lead to spurious disease‐marker association or mask a true association. Several methods have been proposed to match cases and controls prior to genotyping, using family information or epidemiological data, or using genotype data for a modest number of genetic markers. Here, we describe a genetic similarity score matching (GSM) method for efficient matched analysis of cases and controls in a genome‐wide or large‐scale candidate gene association study. GSM comprises three steps: (1) calculating similarity scores for pairs of individuals using the genotype data; (2) matching sets of cases and controls based on the similarity scores so that matched cases and controls have similar genetic background; and (3) using conditional logistic regression to perform association tests. Through computer simulation we show that GSM correctly controls false‐positive rates and improves power to detect true disease predisposing variants. We compare GSM to genomic control using computer simulations, and find improved power using GSM. We suggest that initial matching of cases and controls prior to genotyping combined with careful re‐matching after genotyping is a method of choice for genome‐wide association studies. Genet. Epidemiol. 33:508–517, 2009. © 2009 Wiley‐Liss, Inc.     

Glutamine and Antioxidants in the Critically Ill Patient

Background: The recent large randomized controlled trial of glutamine and antioxidant supplementation suggested that high‐dose glutamine is associated with increased mortality in critically ill patients with multiorgan failure. The objectives of the present analyses were to reevaluate the effect of supplementation after controlling for baseline covariates and to identify potentially important subgroup effects. Materials and Methods: This study was a post hoc analysis of a prospective factorial 2 × 2 randomized trial conducted in 40 intensive care units in North America and Europe. In total, 1223 mechanically ventilated adult patients with multiorgan failure were randomized to receive glutamine, antioxidants, both glutamine and antioxidants, or placebo administered separate from artificial nutrition. We compared each of the 3 active treatment arms (glutamine alone, antioxidants alone, and glutamine + antioxidants) with placebo on 28‐day mortality. Post hoc, treatment effects were examined within subgroups defined by baseline patient characteristics. Logistic regression was used to estimate treatment effects within subgroups after adjustment for baseline covariates and to identify treatment‐by‐subgroup interactions (effect modification). Results: The 28‐day mortality rates in the placebo, glutamine, antioxidant, and combination arms were 25%, 32%, 29%, and 33%, respectively. After adjusting for prespecified baseline covariates, the adjusted odds ratio of 28‐day mortality vs placebo was 1.5 (95% confidence interval, 1.0–2.1, P = .05), 1.2 (0.8–1.8, P = .40), and 1.4 (0.9–2.0, P = .09) for glutamine, antioxidant, and glutamine plus antioxidant arms, respectively. In the post hoc subgroup analysis, both glutamine and antioxidants appeared most harmful in patients with baseline renal dysfunction. No subgroups suggested reduced mortality with supplements. Conclusions: After adjustment for baseline covariates, early provision of high‐dose glutamine administered separately from artificial nutrition was not beneficial and may be associated with increased mortality in critically ill patients with multiorgan failure. For both glutamine and antioxidants, the greatest potential for harm was observed in patients with multiorgan failure that included renal dysfunction upon study enrollment.     

Variable selection for covariate‐adjusted semiparametric inference in randomized clinical trials

Extensive baseline covariate information is routinely collected on participants in randomized clinical trials, and it is well recognized that a proper covariate‐adjusted analysis can improve the efficiency of inference on the treatment effect. However, such covariate adjustment has engendered considerable controversy, as post hoc selection of covariates may involve subjectivity and may lead to biased inference, whereas prior specification of the adjustment may exclude important variables from consideration. Accordingly, how to select covariates objectively to gain maximal efficiency is of broad interest. We propose and study the use of modern variable selection methods for this purpose in the context of a semiparametric framework, under which variable selection in modeling the relationship between outcome and covariates is separated from estimation of the treatment effect, circumventing the potential for selection bias associated with standard analysis of covariance methods. We demonstrate that such objective variable selection techniques combined with this framework can identify key variables and lead to unbiased and efficient inference on the treatment effect. A critical issue in finite samples is validity of estimators of uncertainty, such as standard errors and confidence intervals for the treatment effect. We propose an approach to estimation of sampling variation of estimated treatment effect and show its superior performance relative to that of existing methods. Copyright © 2012 John Wiley & Sons, Ltd.     

Conditional estimation of sensitivity and specificity from a phase 2 biomarker study allowing early termination for futility

Development of a disease screening biomarker involves several phases. In phase 2 its sensitivity and specificity is compared with established thresholds for minimally acceptable performance. Since we anticipate that most candidate markers will not prove to be useful and availability of specimens and funding is limited, early termination of a study is appropriate, if accumulating data indicate that the marker is inadequate. Yet, for markers that complete phase 2, we seek estimates of sensitivity and specificity to proceed with the design of subsequent phase 3 studies. We suggest early stopping criteria and estimation procedures that adjust for bias caused by the early termination option. An important aspect of our approach is to focus on properties of estimates conditional on reaching full study enrollment. We propose the conditional‐UMVUE and contrast it with other estimates, including naïve estimators, the well‐studied unconditional‐UMVUE and the mean and median Whitehead‐adjusted estimators. The conditional‐UMVUE appears to be a very good choice. Copyright © 2008 John Wiley & Sons, Ltd.     

Why caution is recommended with post‐hoc individual patient matching for estimation of treatment effect in parallel‐group randomized controlled trials: The case of acute stroke trials

A post‐hoc individual patient matching procedure was recently proposed within the context of parallel group randomized clinical trials (RCTs) as a method for estimating treatment effect. In this paper, we consider a post‐hoc individual patient matching problem within a parallel group RCT as a multi‐objective decision‐making problem focussing on the trade‐off between the quality of individual matches and the overall percentage of matching. Using acute stroke trials as a context, we utilize exact optimization and simulation techniques to investigate a complex relationship between the overall percentage of individual post‐hoc matching, the size of the respective RCT, and the quality of matching on variables highly prognostic for a good functional outcome after stroke, as well as the dispersion in these variables. It is empirically confirmed that a high percentage of individual post‐hoc matching can only be achieved when the differences in prognostic baseline variables between individually matched subjects within the same pair are sufficiently large and that the unmatched subjects are qualitatively different to the matched ones. It is concluded that the post‐hoc individual matching as a technique for treatment effect estimation in parallel‐group RCTs should be exercised with caution because of its propensity to introduce significant bias and reduce validity. If used with appropriate caution and thorough evaluation, this approach can complement other viable alternative approaches for estimating the treatment effect. Copyright © 2013 John Wiley & Sons, Ltd.     

The use of bootstrapping when using propensity‐score matching without replacement: a simulation study

Propensity‐score matching is frequently used to estimate the effect of treatments, exposures, and interventions when using observational data. An important issue when using propensity‐score matching is how to estimate the standard error of the estimated treatment effect. Accurate variance estimation permits construction of confidence intervals that have the advertised coverage rates and tests of statistical significance that have the correct type I error rates. There is disagreement in the literature as to how standard errors should be estimated. The bootstrap is a commonly used resampling method that permits estimation of the sampling variability of estimated parameters. Bootstrap methods are rarely used in conjunction with propensity‐score matching. We propose two different bootstrap methods for use when using propensity‐score matching without replacementand examined their performance with a series of Monte Carlo simulations. The first method involved drawing bootstrap samples from the matched pairs in the propensity‐score‐matched sample. The second method involved drawing bootstrap samples from the original sample and estimating the propensity score separately in each bootstrap sample and creating a matched sample within each of these bootstrap samples. The former approach was found to result in estimates of the standard error that were closer to the empirical standard deviation of the sampling distribution of estimated effects. © 2014 The Authors. Statistics in Medicine Published by John Wiley & Sons, Ltd.     

Effect heterogeneity by a matching covariate in matched case-control studies: a method for graphs-based representation

The authors describe a method for assessing and characterizing effect heterogeneity related to a matching covariate in case-control studies, using an example from veterinary medicine. Data are from a case-control study conducted in Texas during 1997-1998 of 498 pairs of horses with colic and their controls. Horses were matched by veterinarian and by month of examination. The number of matched pairs of cases and controls varied by veterinarian. The authors demonstrate that there is effect heterogeneity related to this characteristic (i.e., cluster size of veterinarians) for the association of colic with certain covariates, using a moving average approach to conditional logistic regression and graphs-based methods. The method described in this report can be applied to examining effect heterogeneity (or effect modification) by any ordered categorical or continuous covariates for which cases have been matched with controls. The method described enables one to understand the pattern of variation across ordered categorical or continuous matching covariates and allows for any shape for this pattern. This method applies to effect modification when causality might be reasonably assumed.     

Analysis of matched case-control data with incomplete strata: applying longitudinal approaches

BACKGROUND: Matched case-control data have a structure that is similar to longitudinal data with correlated outcomes, except for a retrospective sampling scheme. In conditional logistic regression analysis, sets that are incomplete due to missing covariates and sets with identical values of the covariates do not contribute to the estimation; both situations may cause a loss in efficiency. These problems are more severe when sample sizes are small. We evaluated retrospective models for longitudinal data as alternatives in analyzing matched case-control data. METHODS: We conducted simulations to compare the properties of matched case-control data analyses using conditional likelihood and a commonly used longitudinal approach generalized estimating equation (GEE). We simulated scenarios for one-to-one and one-to-two matching designs, each with various sizes of matching strata, with complete and incomplete strata, and with dichotomous and normal exposures. RESULTS AND CONCLUSIONS: The simulations show that the estimates by conditional likelihood and GEE methods are consistent, and a proper coverage was reached for both binary and continuous exposures. The estimates produced by conditional likelihood have greater standard errors than those obtained by GEE. These relative efficiency losses are more substantial when data contain incomplete matched sets and when the data have small sizes of matching strata; these can be improved by including more controls in the strata. These losses of efficiency also increase as the magnitude of the association increases.     

The Impact of Regression to the Mean on Economic Evaluation in Quasi‐Experimental Pre–Post Studies: The Example of Total Knee Replacement Using Data from the Osteoarthritis Initiative

Many treatments are evaluated using quasi‐experimental pre–post studies susceptible to regression to the mean (RTM). Ignoring RTM could bias the economic evaluation. We investigated this issue using the contemporary example of total knee replacement (TKR), a common treatment for end‐stage osteoarthritis of the knee. Data (n = 4796) were obtained from the Osteoarthritis Initiative database, a longitudinal observational study of osteoarthritis. TKR patients (n = 184) were matched to non‐TKR patients, using propensity score matching on the predicted hazard of TKR and exact matching on osteoarthritis severity and health‐related quality of life (HrQoL). The economic evaluation using the matched control group was compared to the standard method of using the pre‐surgery score as the control. Matched controls were identified for 56% of the primary TKRs. The matched control HrQoL trajectory showed evidence of RTM accounting for a third of the estimated QALY gains from surgery using the pre‐surgery HrQoL as the control. Incorporating RTM into the economic evaluation significantly reduced the estimated cost effectiveness of TKR and increased the uncertainty. A generalized ICER bias correction factor was derived to account for RTM in cost‐effectiveness analysis. RTM should be considered in economic evaluations based on quasi‐experimental pre–post studies. Copyright © 2017 John Wiley & Sons, Ltd.     

Augmented generalized estimating equations for improving efficiency and validity of estimation in cluster randomized trials by leveraging cluster-level and individual-level covariates

Recent methodological advances in covariate adjustment in randomized clinical trials have used semiparametric theory to improve efficiency of inferences by incorporating baseline covariates; these methods have focused on independent outcomes. We modify one of these approaches, augmentation of standard estimators, for use within cluster randomized trials in which treatments are assigned to groups of individuals, thereby inducing correlation. We demonstrate the potential for imbalance correction and efficiency improvement through consideration of both cluster-level covariates and individual-level covariates. To improve small-sample estimation, we consider several variance adjustments. We evaluate this approach for continuous and binary outcomes through simulation and apply it to data from a cluster randomized trial of a community behavioral intervention related to HIV prevention in Tanzania.     

Breaking the matches in a paired t-test for community interventions when the number of pairs is small

There is considerable interest in community interventions for health promotion, where the community is the experimental unit. Because such interventions are expensive, the number of experimental units (communities) is usually small. Because of the small number of communities involved, investigators often match treatment and control communities on demographic variables before randomization to minimize the possibility of a bad split. Unfortunately, matching has been shown to decrease the power of the design when the number of pairs is small, unless the matching variable is very highly correlated with the outcome variable (in this case, with change in the health behaviour). We used computer simulation to examine the performance of an approach in which we matched communities but performed an unmatched analysis. If the appropriate matching variables are unknown, and there are fewer than ten pairs, an unmatched design and analysis has the most power. If, however, one prefers a matched design, then for N > 10, power can be increased by performing an unmatched analysis of the matched data. We also discuss a variant of this procedure, in which an unmatched analysis is performed only if the matching ‘did not work’.