Response‐adaptive randomization for clinical trials with adjustment for covariate imbalance

In clinical trials with a small sample size, the characteristics (covariates) of patients assigned to different treatment arms may not be well balanced. This may lead to an inflated type I error rate. This problem can be more severe in trials that use response‐adaptive randomization rather than equal randomization because the former may result in smaller sample sizes for some treatment arms. We have developed a patient allocation scheme for trials with binary outcomes to adjust the covariate imbalance during response‐adaptive randomization. We used simulation studies to evaluate the performance of the proposed design. The proposed design keeps the important advantage of a standard response‐adaptive design, that is to assign more patients to the better treatment arms, and thus it is ethically appealing. On the other hand, the proposed design improves over the standard response‐adaptive design by controlling covariate imbalance between treatment arms, maintaining the nominal type I error rate, and offering greater power. Copyright © 2010 John Wiley & Sons, Ltd.     

Covariate adjustment in randomized controlled trials with dichotomous outcomes increases statistical power and reduces sample size requirements

OBJECTIVE: Randomized controlled trials (RCTs) with dichotomous outcomes may be analyzed with or without adjustment for baseline characteristics (covariates). We studied type I error, power, and potential reduction in sample size with several covariate adjustment strategies. STUDY DESIGN AND SETTING: Logistic regression analysis was applied to simulated data sets (n=360) with different treatment effects, covariate effects, outcome incidences, and covariate prevalences. Treatment effects were estimated with or without adjustment for a single dichotomous covariate. Strategies included always adjusting for the covariate ("prespecified"), or only when the covariate was predictive or imbalanced. RESULTS: We found that the type I error was generally at the nominal level. The power was highest with prespecified adjustment. The potential reduction in sample size was higher with stronger covariate effects (from 3 to 46%, at 50% outcome incidence and covariate prevalence) and independent of the treatment effect. At lower outcome incidences and/or covariate prevalences, the reduction was lower. CONCLUSION: We conclude that adjustment for a predictive baseline characteristic may lead to a potentially important increase in power of analyses of treatment effect. Adjusted analysis should, hence, be considered more often for RCTs with dichotomous outcomes.     

Randomized controlled trials with time-to-event outcomes: how much does prespecified covariate adjustment increase power?

PURPOSE: We evaluated the effects of various strategies of covariate adjustment on type I error, power, and potential reduction in sample size in randomized controlled trials (RCTs) with time-to-event outcomes. METHODS: We used Cox models in simulated data sets with different treatment effects (hazard ratios [HRs] = 1, 1.4, and 1.7), covariate effects (HRs = 1, 2, and 5), covariate prevalences (10% and 50%), and censoring levels (no, low, and high). Treatment and a single covariate were dichotomous. We examined the sample size that gives the same power as an unadjusted analysis for three strategies: prespecified, significant predictive, and significant imbalance. RESULTS: Type I error generally was at the nominal level. The power to detect a true treatment effect was greater with adjusted than unadjusted analyses, especially with prespecified and significant-predictive strategies. Potential reductions in sample size with a covariate HR between 2 and 5 were between 15% and 44% (covariate prevalence 50%) and between 4% and 12% (covariate prevalence 10%). The significant-imbalance strategy yielded small reductions. The reduction was greater with stronger covariate effects, but was independent of treatment effect, sample size, and censoring level. CONCLUSIONS: Adjustment for one predictive baseline characteristic yields greater power to detect a true treatment effect than unadjusted analysis, without inflation of type I error and with potentially moderate reductions in sample size. Analysis of RCTs with time-to-event outcomes should adjust for predictive covariates.     

Methods for covariate adjustment in cost-effectiveness analysis that use cluster randomised trials

Statistical methods have been developed for cost-effectiveness analyses of cluster randomised trials (CRTs) where baseline covariates are balanced. However, CRTs may show systematic differences in individual and cluster-level covariates between the treatment groups. This paper presents three methods to adjust for imbalances in observed covariates: seemingly unrelated regression with a robust standard error, a 'two-stage' bootstrap approach combined with seemingly unrelated regression and multilevel models. We consider the methods in a cost-effectiveness analysis of a CRT with covariate imbalance, unequal cluster sizes and a prognostic relationship that varied by treatment group. The cost-effectiveness results differed according to the approach for covariate adjustment. A simulation study then assessed the relative performance of methods for addressing systematic imbalance in baseline covariates. The simulations extended the case study and considered scenarios with different levels of confounding, cluster size variation and few clusters. Performance was reported as bias, root mean squared error and CI coverage of the incremental net benefit. Even with low levels of confounding, unadjusted methods were biased, but all adjusted methods were unbiased. Multilevel models performed well across all settings, and unlike the other methods, reported CI coverage close to nominal levels even with few clusters of unequal sizes.     

An evaluation of constrained randomization for the design and analysis of group‐randomized trials with binary outcomes

Group‐randomized trials are randomized studies that allocate intact groups of individuals to different comparison arms. A frequent practical limitation to adopting such research designs is that only a limited number of groups may be available, and therefore, simple randomization is unable to adequately balance multiple group‐level covariates between arms. Therefore, covariate‐based constrained randomization was proposed as an allocation technique to achieve balance. Constrained randomization involves generating a large number of possible allocation schemes, calculating a balance score that assesses covariate imbalance, limiting the randomization space to a prespecified percentage of candidate allocations, and randomly selecting one scheme to implement. When the outcome is binary, a number of statistical issues arise regarding the potential advantages of such designs in making inference. In particular, properties found for continuous outcomes may not directly apply, and additional variations on statistical tests are available. Motivated by two recent trials, we conduct a series of Monte Carlo simulations to evaluate the statistical properties of model‐based and randomization‐based tests under both simple and constrained randomization designs, with varying degrees of analysis‐based covariate adjustment. Our results indicate that constrained randomization improves the power of the linearization F‐test, the KC‐corrected GEE t‐test (Kauermann and Carroll, 2001, Journal of the American Statistical Association 96, 1387‐1396), and two permutation tests when the prognostic group‐level variables are controlled for in the analysis and the size of randomization space is reasonably small. We also demonstrate that constrained randomization reduces power loss from redundant analysis‐based adjustment for non‐prognostic covariates. Design considerations such as the choice of the balance metric and the size of randomization space are discussed.     

A comparison of the results of intent‐to‐treat,per‐protocol,and g‐estimation in the presence of non‐random treatment changes in a time‐to‐event non‐inferiority trial

While intent‐to‐treat (ITT) analysis is widely accepted for superiority trials, there remains debate about its role in non‐inferiority trials. It has often been said that ITT analysis tends to be anti‐conservative in demonstrating non‐inferiority, suggesting that per‐protocol (PP) analysis may be preferable for non‐inferiority trials, despite the inherent bias of such analyses. We propose using randomization‐based g‐estimation analyses that more effectively preserve the integrity of randomization than do the more widely used PP analyses. Simulation studies were conducted to investigate the impacts of different types of treatment changes on the conservatism or anti‐conservatism of analyses using the ITT, PP, and g‐estimation methods in a time‐to‐event outcome. The ITT results were anti‐conservative for all simulations. Anti‐conservativeness increased with the percentage of treatment change and was more pronounced for outcome‐dependent treatment changes. PP analysis, in which treatment‐switching cases were censored at the time of treatment change, maintained type I error near the nominal level for independent treatment changes, whereas for outcome‐dependent cases, PP analysis was either conservative or anti‐conservative depending on the mechanism underlying the percentage of treatment changes. G‐estimation analysis maintained type I error near the nominal level even for outcome‐dependent treatment changes, although information on unmeasured covariates is not used in the analysis. Thus, randomization‐based g‐estimation analyses should be used to supplement the more conventional ITT and PP analyses, especially for non‐inferiority trials. Copyright © 2010 John Wiley & Sons, Ltd.     

Testing for imbalance of covariates in controlled experiments

Results of a controlled experiment are often adjusted for covariates found by a preliminary test to differ significantly between the treatment and control groups. The resulting test's true significance level is lower than the nominal level. Greater power can be achieved by always adjusting for a covariate that is highly correlated with the response regardless of its distribution between groups.     

Interaction of treatment with a continuous variable: simulation study of significance level for several methods of analysis

Interactions between treatments and covariates in RCTs are a key topic. Standard methods for modelling treatment–covariate interactions with continuous covariates are categorisation or linear functions. Both approaches are easily criticised, but for different reasons. Multivariable fractional polynomial interactions, an approach based on fractional polynomials with the linear interaction model as the simplest special case, was proposed. Four variants of multivariable fractional polynomial interaction (FLEX1–FLEX4), allowing varying flexibility in functional form, were suggested. However, their properties are unknown, and comparisons with other procedures are unavailable. Additionally, we consider various methods based on categorisation and on cubic regression splines. We present the results of a simulation study to determine the significance level (probability of a type 1 error) of various tests for interaction between a binary covariate (‘treatment effect’) and a continuous covariate in univariate analysis. We consider a simplified setting in which the response variable is conditionally normally distributed, given the continuous covariate. We consider two main cases with the covariate distribution well behaved (approximately symmetric) or badly behaved (positively skewed). We construct nine scenarios with different functional forms for the main effect. In the well‐behaved case, significance levels are in general acceptably close to nominal and are slightly better for the larger sample size (n = 250 and 500 were investigated). In the badly behaved case, departures from nominal are more pronounced for several approaches. For a final assessment of these results and recommendations for practice, a study of power is needed. Copyright © 2013 John Wiley & Sons, Ltd.     

Managing competing demands in the implementation of response‐adaptive randomization in a large multicenter phase III acute stroke trial

It is well known that competing demands exist between the control of important covariate imbalance and protection of treatment allocation randomness in confirmative clinical trials. When implementing a response‐adaptive randomization algorithm in confirmative clinical trials designed under a frequentist framework, additional competing demands emerge between the shift of the treatment allocation ratio and the preservation of the power. Based on a large multicenter phase III stroke trial, we present a patient randomization scheme that manages these competing demands by applying a newly developed minimal sufficient balancing design for baseline covariates and a cap on the treatment allocation ratio shift in order to protect the allocation randomness and the power. Statistical properties of this randomization plan are studied by computer simulation. Trial operation characteristics, such as patient enrollment rate and primary outcome response delay, are also incorporated into the randomization plan. Copyright © 2014 John Wiley & Sons, Ltd.     

An evaluation of constrained randomization for the design and analysis of group‐randomized trials

In group‐randomized trials, a frequent practical limitation to adopting rigorous research designs is that only a small number of groups may be available, and therefore, simple randomization cannot be relied upon to balance key group‐level prognostic factors across the comparison arms. Constrained randomization is an allocation technique proposed for ensuring balance and can be used together with a permutation test for randomization‐based inference. However, several statistical issues have not been thoroughly studied when constrained randomization is considered. Therefore, we used simulations to evaluate key issues including the following: the impact of the choice of the candidate set size and the balance metric used to guide randomization; the choice of adjusted versus unadjusted analysis; and the use of model‐based versus randomization‐based tests. We conducted a simulation study to compare the type I error and power of the F‐test and the permutation test in the presence of group‐level potential confounders. Our results indicate that the adjusted F‐test and the permutation test perform similarly and slightly better for constrained randomization relative to simple randomization in terms of power, and the candidate set size does not substantially affect their power. Under constrained randomization, however, the unadjusted F‐test is conservative, while the unadjusted permutation test carries the desired type I error rate as long as the candidate set size is not too small; the unadjusted permutation test is consistently more powerful than the unadjusted F‐test and gains power as candidate set size changes. Finally, we caution against the inappropriate specification of permutation distribution under constrained randomization. An ongoing group‐randomized trial is used as an illustrative example for the constrained randomization design. Copyright © 2015 John Wiley & Sons, Ltd.     

A comparison of permutation and mixed-model regression methods for the analysis of simulated data in the context of a group-randomized trial

Our first purpose was to determine whether, in the context of a group-randomized trial (GRT) with Gaussian errors, permutation or mixed-model regression methods fare better in the presence of measurable confounding in terms of their Monte Carlo type I error rates and power. Our results indicate that given a proper randomization, the type I error rate is similar for both methods, whether unadjusted or adjusted, even in small studies. However, our results also show that should the investigator face the unfortunate circumstance in which modest confounding exists in the only realization available, the unadjusted analysis risks a type I error; in this regard, there was little to distinguish the two methods. Finally, our results show that power is similar for the two methods and, not surprisingly, better for the adjusted tests.Our second purpose was to examine the relative performance of permutation and mixed-model regression methods in the context of a GRT when the normality assumptions underlying the mixed model are violated. Published studies have examined the impact of violation of this assumption at the member level only. Our findings indicate that both methods perform well when the assumption is violated so long as the ICC is very small and the design is balanced at the group level. However, at ICC>or=0.01, the permutation test carries the nominal type I error rate while the model-based test is conservative and so less powerful. Binomial group- and member-level errors did not otherwise change the relative performance of the two methods with regard to confounding.     

Robust extraction of covariate information to improve estimation efficiency in randomized trials

In randomized trials, investigators typically rely upon an unadjusted estimate of the mean outcome within each treatment arm to draw causal inferences. Statisticians have underscored the gain in efficiency that can be achieved from covariate adjustment in randomized trials with a focus on problems involving linear models. Despite recent theoretical advances, there has been a reluctance to adjust for covariates based on two primary reasons: (i) covariate-adjusted estimates based on conditional logistic regression models have been shown to be less precise and (ii) concern over the opportunity to manipulate the model selection process for covariate adjustments to obtain favorable results. In this paper, we address these two issues and summarize recent theoretical results on which is based a proposed general methodology for covariate adjustment under the framework of targeted maximum likelihood estimation in trials with two arms where the probability of treatment is 50%. The proposed methodology provides an estimate of the true causal parameter of interest representing the population-level treatment effect. It is compared with the estimates based on conditional logistic modeling, which only provide estimates of subgroup-level treatment effects rather than marginal (unconditional) treatment effects. We provide a clear criterion for determining whether a gain in efficiency can be achieved with covariate adjustment over the unadjusted method. We illustrate our strategy using a resampled clinical trial dataset from a placebo controlled phase 4 study. Results demonstrate that gains in efficiency can be achieved even with binary outcomes through covariate adjustment leading to increased statistical power.     

Covariate imbalance and random allocation in clinical trials

A model is developed to estimate the effect of covariate imbalance on the size of a test of treatment efficacy in randomized clinical trials comparing two treatments when dispersion parameters are known. It is concluded that tests of homogeneity on the covariates should not be performed, that covariate imbalance is just as much a problem for large studies as for small ones in terms of effect on size, and that the effect of correlation between covariates and measures of efficacy is more complex than has previously been suggested. The best way to adjust for covariate imbalance is by an analysis of covariance.     

Improve efficiency and reduce bias of Cox regression models for two‐stage randomization designs using auxiliary covariates

Two‐stage randomization designs are broadly accepted and becoming increasingly popular in clinical trials for cancer and other chronic diseases to assess and compare the effects of different treatment policies. In this paper, we propose an inferential method to estimate the treatment effects in two‐stage randomization designs, which can improve the efficiency and reduce bias in the presence of chance imbalance of a robust covariate‐adjustment without additional assumptions required by Lokhnygina and Helterbrand (Biometrics, 63:422‐428)'s inverse probability weighting (IPW) method. The proposed method is evaluated and compared with the IPW method using simulations and an application to data from an oncology clinical trial. Given the predictive power of baseline covariates collected in this real data, our proposed method obtains 17–38% gains in efficiency compared with the IPW method in terms of overall survival outcome. Copyright © 2017 John Wiley & Sons, Ltd.     

ON DESIGN CONSIDERATIONS AND RANDOMIZATION-BASED INFERENCE FOR COMMUNITY INTERVENTION TRIALS

This paper discusses design considerations and the role of randomization-based inference in randomized community intervention trials. We stress that longitudinal follow-up of cohorts within communities often yields useful information on the effects of intervention on individuals, whereas cross-sectional surveys can usefully assess the impact of intervention on group indices of health. We also discuss briefly special design considerations, such as sampling cohorts from targeted subpopulations (for example, heavy smokers), matching the communities, calculating sample size, and other practical issues. We present randomization tests for matched and unmatched cohort designs. As is well known, these tests necessarily have proper size under the strong null hypothesis that treatment has no effect on any community response. It is less well known, however, that the size of randomization tests can exceed nominal levels under the ‘weak’ null hypothesis that intervention does not affect the average community response. Because this weak null hypothesis is of interest in community intervention trials, we study the size of randomization tests by simulation under conditions in which the weak null hypothesis holds but the strong null hypothesis does not. In unmatched studies, size may exceed nominal levels under the weak null hypothesis if there are more intervention than control communities and if the variance among community responses is larger among control communities than among intervention communities; size may also exceed nominal levels if there are more control than intervention communities and if the variance among community responses is larger among intervention communities. Otherwise, size is likely near nominal levels. To avoid such problems, we recommend use of the same numbers of control and intervention communities in unmatched designs. Pair-matched designs usually have size near nominal levels, even under the weak null hypothesis. We have identified some extreme cases, unlikely to arise in practice, in which even the size of pair-matched studies can exceed nominal levels. These simulations, however, tend to confirm the robustness of randomization tests for matched and unmatched community intervention trials, particularly if the latter designs have equal numbers of intervention and control communities. We also describe adaptations of randomization tests to allow for covariate adjustment, missing data, and application to cross-sectional surveys. We show that covariate adjustment can increase power, but such power gains diminish as the random component of variation among communities increases, which corresponds to increasing intraclass correlation of responses within communities. We briefly relate our results to model-based methods of inference for community intervention trials that include hierarchical models such as an analysis of variance model with random community effects and fixed intervention effects. Although we have tailored this paper to the design of community intervention trials, many of the ideas apply to other experiments in which one allocates groups or clusters of subjects at random to intervention or control treatments.     

Increasing the power of the Mann‐Whitney test in randomized experiments through flexible covariate adjustment

The Mann‐Whitney U test is frequently used to evaluate treatment effects in randomized experiments with skewed outcome distributions or small sample sizes. It may lack power, however, because it ignores the auxiliary baseline covariate information that is routinely collected. Wald and score tests in so‐called probabilistic index models generalize the Mann‐Whitney U test to enable adjustment for covariates, but these may lack robustness by demanding correct model specification and do not lend themselves to small sample inference. Using semiparametric efficiency theory, we here propose an alternative extension of the Mann‐Whitney U test, which increases its power by exploiting covariate information in an objective way and which lends itself to permutation inference. Simulation studies and an application to an HIV clinical trial show that the proposed permutation test attains the nominal Type I error rate and can be drastically more powerful than the classical Mann‐Whitney U test. Copyright © 2014 John Wiley & Sons, Ltd.     

Ordered-subset analysis (OSA) for family-based association mapping of complex traits

Association analysis provides a powerful tool for complex disease gene mapping. However, in the presence of genetic heterogeneity, the power for association analysis can be low since only a fraction of the collected families may carry a specific disease susceptibility allele. Ordered-subset analysis (OSA) is a linkage test that can be powerful in the presence of genetic heterogeneity. OSA uses trait-related covariates to identify a subset of families that provide the most evidence for linkage. A similar strategy applied to genetic association analysis would likely result in increased power to detect association. Association in the presence of linkage (APL) is a family-based association test (FBAT) for nuclear families with multiple affected siblings that properly infers missing parental genotypes when linkage is present. We propose here APL-OSA, which applies the OSA method to the APL statistic to identify a subset of families that provide the most evidence for association. A permutation procedure is used to approximate the distribution of the APL-OSA statistic under the null hypothesis that there is no relationship between the family-specific covariate and the family-specific evidence for allelic association. We performed a comprehensive simulation study to verify that APL-OSA has the correct type I error rate under the null hypothesis. This simulation study also showed that APL-OSA can increase power relative to other commonly used association tests (APL, FBAT and FBAT with covariate adjustment) in the presence of genetic heterogeneity. Finally, we applied APL-OSA to a family study of age-related macular degeneration, where cigarette smoking was used as a covariate.     

A quantitative study of the bias in estimating the treatment effect caused by omitting a balanced covariate in survival models

This paper discusses the quantitative aspects of bias in estimates of treatment effect in survival models when there is failure to adjust on balanced prognostic variables. A simple numerical example of this bias is given along with approximate formulae for its calculation in the multiplicative exponential survival model. The accuracy of the formulae is checked by simulation. In addition, approximate calculations and simulations of power loss and the effects of omitting more than one prognostic covariate are presented. The Weibull and Cox models are also examined using simulation. Study of this bias is pertinent to much applied work, and shows that the effect of omitting balanced covariates can be modest unless the variables are strongly prognostic or many in number. This work emphasizes the need for thorough comparisons of adjusted and unadjusted analyses for sensible interpretation of treatment effects.     

Increasing the sample size when the unblinded interim result is promising

Increasing the sample size based on unblinded interim result may inflate the type I error rate and appropriate statistical adjustments may be needed to control the type I error rate at the nominal level. We briefly review the existing approaches which allow early stopping due to futility, or change the test statistic by using different weights, or adjust the critical value for final test, or enforce rules for sample size recalculation. The implication of early stopping due to futility and a simple modification to the weighted Z-statistic approach are discussed. In this paper, we show that increasing the sample size when the unblinded interim result is promising will not inflate the type I error rate and therefore no statistical adjustment is necessary. The unblinded interim result is considered promising if the conditional power is greater than 50 per cent or equivalently, the sample size increment needed to achieve a desired power does not exceed an upper bound. The actual sample size increment may be determined by important factors such as budget, size of the eligible patient population and competition in the market. The 50 per cent-conditional-power approach is extended to a group sequential trial with one interim analysis where a decision may be made at the interim analysis to stop the trial early due to a convincing treatment benefit, or to increase the sample size if the interim result is not as good as expected. The type I error rate will not be inflated if the sample size may be increased only when the conditional power is greater than 50 per cent. If there are two or more interim analyses in a group sequential trial, our simulation study shows that the type I error rate is also well controlled.     

An approximate approach to sample size determination in bioequivalence testing with multiple pharmacokinetic responses

The approval of generic drugs requires the evidence of average bioequivalence (ABE) on both the area under the concentration–time curve and the peak concentration C_max. The bioequivalence (BE) hypothesis can be decomposed into the non‐inferiority (NI) and non‐superiority (NS) hypothesis. Most of regulatory agencies employ the two one‐sided tests (TOST) procedure to test ABE between two formulations. As it is based on the intersection–union principle, the TOST procedure is conservative in terms of the type I error rate. However, the type II error rate is the sum of the type II error rates with respect to each null hypothesis of NI and NS hypotheses. When the difference in population means between two treatments is not 0, no close‐form solution for the sample size for the BE hypothesis is available. Current methods provide the sample sizes with either insufficient power or unnecessarily excessive power. We suggest an approximate method for sample size determination, which can also provide the type II rate for each of NI and NS hypotheses. In addition, the proposed method is flexible to allow extension from one pharmacokinetic (PK) response to determination of the sample size required for multiple PK responses. We report the results of a numerical study. An R code is provided to calculate the sample size for BE testing based on the proposed methods. Copyright © 2014 John Wiley & Sons, Ltd.