Adaptive extensions of a two-stage group sequential procedure for testing primary and secondary endpoints (II): sample size re-estimation

In this part II of the paper on adaptive extensions of a two‐stage group sequential procedure (GSP) for testing primary and secondary endpoints, we focus on the second stage sample size re‐estimation based on the first stage data. First, we show that if we use the Cui–Huang–Wang statistics at the second stage, then we can use the same primary and secondary boundaries as for the original procedure (without sample size re‐estimation) and still control the type I familywise error rate. This extends their result for the single endpoint case. We further show that the secondary boundary can be sharpened in this case by taking the unknown correlation coefficient ρ between the primary and secondary endpoints into account through the use of the confidence limit method proposed in part I of this paper. If we use the sufficient statistics instead of the CHW statistics, then we need to modify both the primary and secondary boundaries; otherwise, the error rate can get inflated. We show how to modify the boundaries of the original group sequential procedure to control the familywise error rate. We provide power comparisons between competing procedures. We illustrate the procedures with a clinical trial example. Copyright © 2012 John Wiley & Sons, Ltd.     

Blinded sample size reassessment in non-inferiority and equivalence trials

Even in situations where the design and conduct of clinical trials is highly standardized, there may be a considerable between-study variation in the observed variability of the primary outcome variable. As a consequence, performing a study in a fixed sample size design implies a considerable risk of resulting in a too high or too low sample size. This difficulty can be alleviated by applying a design with internal pilot study. After a provisional sample size calculation in the planning stage, a portion of the planned sample is recruited and the sample size is recalculated on the basis of the observed variability. To comply with the requirement of some regulatory guidelines only blinded data should be used for the reassessment procedure. Furthermore, the effect on the type I error rate should be quantified. The current literature presents analytical results on the actual level in the t-test situation only for superiority trials. In these situations, blinded sample size recalculation does not lead to an inflation of the type I error rate. We extended the methodology to non-inferiority and equivalence trials with normally distributed outcome variable and hypotheses formulated in terms of the ratio and difference of means. Surprisingly, in contrast to the case of testing superiority, we observed actual type I error rates above the nominal level. The extent of inflation depends on the required sample size, the sample size of the internal pilot study, and the standardized equivalence or non-inferiority margin. It turned out that the elevation of the significance level is negligible for most practical situations. Nevertheless, the consequences of sample size reassessment have to be discussed case by case and regulatory concerns with respect to the actual size of the procedure cannot generally be refuted by referring to the fact that only blinded data were used.     

Flexible selection of a single treatment incorporating short‐term endpoint information in a phase II/III clinical trial

Seamless phase II/III clinical trials in which an experimental treatment is selected at an interim analysis have been the focus of much recent research interest. Many of the methods proposed are based on the group sequential approach. This paper considers designs of this type in which the treatment selection can be based on short‐term endpoint information for more patients than have primary endpoint data available. We show that in such a case, the familywise type I error rate may be inflated if previously proposed group sequential methods are used and the treatment selection rule is not specified in advance. A method is proposed to avoid this inflation by considering the treatment selection that maximises the conditional error given the data available at the interim analysis. A simulation study is reported that illustrates the type I error rate inflation and compares the power of the new approach with two other methods: a combination testing approach and a group sequential method that does not use the short‐term endpoint data, both of which also strongly control the type I error rate. The new method is also illustrated through application to a study in Alzheimer's disease. © 2015 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.     

A confirmatory seamless phase II/III clinical trial design incorporating short‐term endpoint information

Seamless phase II/III designs allow strong control of the familywise type I error rate when the most promising of a number of experimental treatments is selected at an interim analysis to continue along with the control treatment. If the primary endpoint is observed only after long‐term follow‐up it may be desirable to use correlated short‐term endpoint data available at the interim analysis to inform the treatment selection. If short‐term data are available for some patients for whom the primary endpoint is not available, basing treatment selection on these data may, however, lead to inflation of the type I error rate. This paper proposes a method for the adjustment of the usual group‐sequential boundaries to maintain strong control of the familywise error rate even when short‐term endpoint data are used for the treatment selection at the first interim analysis. This method allows the use of the short‐term data, leading to an increase in power when these data are correlated with the primary endpoint data. Copyright © 2010 John Wiley & Sons, Ltd.     

A method to estimate the variance of an endpoint from an on-going blinded trial

Blinded estimation of variance allows for changing the sample size without compromising the integrity of the trial. Some of the methods that estimate the variance in a blinded manner either make untenable assumptions or are only applicable to two-treatment trials. We propose a new method for continuous endpoints that makes minimal assumptions. The method uses the enrollment order of subjects and the randomization block size to estimate the variance. It can be applied to normal or non-normal data, trials with two or more treatments, equal or unequal allocation schemes, fixed or random randomization block sizes, and single or multi-centre trials. The variance estimator is unbiased and performs best when the randomization block size is the smallest. Simulation results suggest that for many commonly used randomization block sizes the proposed estimator is expected to perform well. The proposed method is used to estimate the variance of the endpoint for two trials and is shown to perform well by comparison with its unblinded counterpart.     

Blinded sample size reestimation with count data: Methods and applications in multiple sclerosis

Sample size estimation in clinical trials depends critically on nuisance parameters, such as variances or overall event rates, which have to be guessed or estimated from previous studies in the planning phase of a trial. Blinded sample size reestimation estimates these nuisance parameters based on blinded data from the ongoing trial, and allows to adjust the sample size based on the acquired information. In the present paper, this methodology is developed for clinical trials with count data as the primary endpoint. In multiple sclerosis such endpoints are commonly used in phase 2 trials (lesion counts in magnetic resonance imaging (MRI)) and phase 3 trials (relapse counts). Sample size adjustment formulas are presented for both Poisson‐distributed data and for overdispersed Poisson‐distributed data. The latter arise from sometimes considerable between‐patient heterogeneity, which can be observed in particular in MRI lesion counts. The operation characteristics of the procedure are evaluated by simulations and recommendations on how to choose the size of the internal pilot study are given. The results suggest that blinded sample size reestimation for count data maintains the required power without an increase in the type I error. Copyright © 2010 John Wiley & Sons, Ltd.     

Some comments on frequently used multiple endpoint adjustment methods in clinical trials

Confirmatory clinical trials often classify clinical response variables into primary and secondary endpoints. The presence of two or more primary endpoints in a clinical trial usually means that some adjustments of the observed p-values for multiplicity of tests may be required for the control of the type I error rate. In this paper, we discuss statistical concerns associated with some commonly used multiple endpoint adjustment procedures. We also present limited Monte Carlo simulation results to demonstrate the performance of selected p-value-based methods in protecting the type I error rate. © 1997 by John Wiley & Sons, Ltd.     

Sample size determination in group‐sequential clinical trials with two co‐primary endpoints

We discuss sample size determination in group‐sequential designs with two endpoints as co‐primary. We derive the power and sample size within two decision‐making frameworks. One is to claim the test intervention's benefit relative to control when superiority is achieved for the two endpoints at the same interim timepoint of the trial. The other is when superiority is achieved for the two endpoints at any interim timepoint, not necessarily simultaneously. We evaluate the behaviors of sample size and power with varying design elements and provide a real example to illustrate the proposed sample size methods. In addition, we discuss sample size recalculation based on observed data and evaluate the impact on the power and Type I error rate. Copyright © 2014 John Wiley & Sons, Ltd.     

Application of a stopping rule based on total treatment failures: the postoperative Crohn's disease trial.

The Postoperative Crohn's Disease Trial (PCDT), a placebo-controlled randomized trial of Rowasa I in the prevention of postoperative recurrence of Crohn's disease, is used as an example of how a stopping rule based on total endpoint occurrences can provide considerable advantage over standard fixed sample size methods. It can be used when the primary outcome is occurrence or time to occurrence and does not raise the troublesome issues regarding the unblinding of group differences that other sequential methods create. The main advantage of the total endpoint stopping rule is that it provides set power. Standard fixed sample size designs provide a given power only on average. The power actually achieved in a particular fixed sample size trial is largely determined by the overall observed rate of endpoint occurrences. This claim about the total endpoint stopping rule is well established in the statistical literature and, as well as outlining the mathematical details in an Appendix, we use computer simulation of the PCDT to demonstrate that use of the stopping rule will allow termination of the trial while maintaining power and type I error at a predetermined level.     

Adaptive seamless designs with interim treatment selection: a case study in oncology

The planning of an oncology clinical trial with a seamless phase II/III adaptive design is discussed. Two regimens of an experimental treatment are compared to a control at an interim analysis, and the most‐promising regimen is selected to continue, together with control, until the end of the study. Because the primary endpoint is expected to be immature at the interim regimen selection analysis, designs that incorporate primary as well as surrogate endpoints in the regimen selection process are considered. The final testing of efficacy at the end of the study comparing the selected regimen to the control with respect to the primary endpoint uses all relevant data collected both before and after the regimen selection analysis. Several approaches for testing the primary hypothesis are assessed with regard to power and type I error rate. Because the operating characteristics of these designs depend on the specific regimen selection rules considered, benchmark scenarios are proposed in which a perfect surrogate and no surrogate is used at the regimen selection analysis. The operating characteristics of these benchmark scenarios provide a range where those of the actual study design are expected to lie. A discussion on family‐wise error rate control for testing primary and key secondary endpoints as well as an assessment of bias in the final treatment effect estimate for the selected regimen are also presented. Copyright © 2015 John Wiley & Sons, Ltd.     

Re-estimating the sample size of an on-going blinded trial based on the method of randomization block sums

We extend a method we had previously described (Statist. Med. 2005) to estimate the within-group variance of a continuous endpoint without breaking the blind in a randomized clinical trial. Specifically, we: (a) explain how the method may be used for a wider set of designs than we had previously indicated; (b) obtain a within-group, covariate-adjusted, blinded variance estimator; (c) illustrate use of the method for sample size re-estimation; and (d) describe a procedure to determine whether or not the blinded variance estimator works well not just on average but for the data set at hand. The proposed method is simple to use and makes no additional assumptions than is made for unblinded analysis. Simulations show that for realistic sample sizes there is virtually no inflation in the Type I error rate. When weighing the burden imposed by interim unblinded re-estimation with the loss in precision with blinded re-estimation, it may be advantageous for some trials to use the blinded method.     

Controlling the type I error rate in two‐stage sequential adaptive designs when testing for average bioequivalence

In a 2×2 crossover trial for establishing average bioequivalence (ABE) of a generic agent and a currently marketed drug, the recommended approach to hypothesis testing is the two one‐sided test (TOST) procedure, which depends, among other things, on the estimated within‐subject variability. The power of this procedure, and therefore the sample size required to achieve a minimum power, depends on having a good estimate of this variability. When there is uncertainty, it is advisable to plan the design in two stages, with an interim sample size reestimation after the first stage, using an interim estimate of the within‐subject variability. One method and 3 variations of doing this were proposed by Potvin et al. Using simulation, the operating characteristics, including the empirical type I error rate, of the 4 variations (called Methods A, B, C, and D) were assessed by Potvin et al and Methods B and C were recommended. However, none of these 4 variations formally controls the type I error rate of falsely claiming ABE, even though the amount of inflation produced by Method C was considered acceptable. A major disadvantage of assessing type I error rate inflation using simulation is that unless all possible scenarios for the intended design and analysis are investigated, it is impossible to be sure that the type I error rate is controlled. Here, we propose an alternative, principled method of sample size reestimation that is guaranteed to control the type I error rate at any given significance level. This method uses a new version of the inverse‐normal combination of p‐values test, in conjunction with standard group sequential techniques, that is more robust to large deviations in initial assumptions regarding the variability of the pharmacokinetic endpoints. The sample size reestimation step is based on significance levels and power requirements that are conditional on the first‐stage results. This necessitates a discussion and exploitation of the peculiar properties of the power curve of the TOST testing procedure. We illustrate our approach with an example based on a real ABE study and compare the operating characteristics of our proposed method with those of Method B of Povin et al.     

Adaptive extensions of a two-stage group sequential procedure for testing primary and secondary endpoints (I): unknown correlation between the endpoints

In a previous paper we studied a two‐stage group sequential procedure (GSP) for testing primary and secondary endpoints where the primary endpoint serves as a gatekeeper for the secondary endpoint. We assumed a simple setup of a bivariate normal distribution for the two endpoints with the correlation coefficient ρ between them being either an unknown nuisance parameter or a known constant. Under the former assumption, we used the least favorable value of ρ = 1 to compute the critical boundaries of a conservative GSP. Under the latter assumption, we computed the critical boundaries of an exact GSP. However, neither assumption is very practical. The ρ = 1 assumption is too conservative resulting in loss of power, whereas the known ρ assumption is never true in practice. In this part I of a two‐part paper on adaptive extensions of this two‐stage procedure (part II deals with sample size re‐estimation), we propose an intermediate approach that uses the sample correlation coefficient r from the first‐stage data to adaptively adjust the secondary boundary after accounting for the sampling error in r via an upper confidence limit on ρ by using a method due to Berger and Boos. We show via simulation that this approach achieves 5–11% absolute secondary power gain for ρ ≤0.5. The preferred boundary combination in terms of high primary as well as secondary power is that of O'Brien and Fleming for the primary and of Pocock for the secondary. The proposed approach using this boundary combination achieves 72–84% relative secondary power gain (with respect to the exact GSP that assumes known ρ). We give a clinical trial example to illustrate the proposed procedure. Copyright © 2012 John Wiley & Sons, Ltd.     

Design and semiparametric analysis of non‐inferiority trials with active and placebo control for censored time‐to‐event data

The clinical trial design including a test treatment, an active control and a placebo is called the gold standard design. In this paper, we develop a statistical method for planning and evaluating non‐inferiority trials with gold standard design for right‐censored time‐to‐event data. We consider both lost to follow‐up and administrative censoring. We present a semiparametric approach that only assumes the proportionality of the hazard functions. In particular, we develop an algorithm for calculating the minimal total sample size and its optimal allocation to treatment groups such that a desired power can be attained for a specific parameter constellation under the alternative. For the purpose of sample size calculation, we assume the endpoints to be Weibull distributed. By means of simulations, we investigate the actual type I error rate, power and the accuracy of the calculated sample sizes. Finally, we compare our procedure with a previously proposed procedure assuming exponentially distributed event times. To illustrate our method, we consider a double‐blinded, randomized, active and placebo controlled trial in major depression. Copyright © 2013 John Wiley & Sons, Ltd.     

Blinded sample size re‐estimation in three‐arm trials with ‘gold standard’ design

In this article, we study blinded sample size re‐estimation in the ‘gold standard’ design with internal pilot study for normally distributed outcomes. The ‘gold standard’ design is a three‐arm clinical trial design that includes an active and a placebo control in addition to an experimental treatment. We focus on the absolute margin approach to hypothesis testing in three‐arm trials at which the non‐inferiority of the experimental treatment and the assay sensitivity are assessed by pairwise comparisons. We compare several blinded sample size re‐estimation procedures in a simulation study assessing operating characteristics including power and type I error. We find that sample size re‐estimation based on the popular one‐sample variance estimator results in overpowered trials. Moreover, sample size re‐estimation based on unbiased variance estimators such as the Xing–Ganju variance estimator results in underpowered trials, as it is expected because an overestimation of the variance and thus the sample size is in general required for the re‐estimation procedure to eventually meet the target power. To overcome this problem, we propose an inflation factor for the sample size re‐estimation with the Xing–Ganju variance estimator and show that this approach results in adequately powered trials. Because of favorable features of the Xing–Ganju variance estimator such as unbiasedness and a distribution independent of the group means, the inflation factor does not depend on the nuisance parameter and, therefore, can be calculated prior to a trial. Moreover, we prove that the sample size re‐estimation based on the Xing–Ganju variance estimator does not bias the effect estimate. Copyright © 2017 John Wiley & Sons, Ltd.     

Response‐adaptive randomization for clinical trials with adjustment for covariate imbalance

In clinical trials with a small sample size, the characteristics (covariates) of patients assigned to different treatment arms may not be well balanced. This may lead to an inflated type I error rate. This problem can be more severe in trials that use response‐adaptive randomization rather than equal randomization because the former may result in smaller sample sizes for some treatment arms. We have developed a patient allocation scheme for trials with binary outcomes to adjust the covariate imbalance during response‐adaptive randomization. We used simulation studies to evaluate the performance of the proposed design. The proposed design keeps the important advantage of a standard response‐adaptive design, that is to assign more patients to the better treatment arms, and thus it is ethically appealing. On the other hand, the proposed design improves over the standard response‐adaptive design by controlling covariate imbalance between treatment arms, maintaining the nominal type I error rate, and offering greater power. Copyright © 2010 John Wiley & Sons, Ltd.     

Inflation of the type I error rate when a continuous confounding variable is categorized in logistic regression analyses

This paper demonstrates an inflation of the type I error rate that occurs when testing the statistical significance of a continuous risk factor after adjusting for a correlated continuous confounding variable that has been divided into a categorical variable. We used Monte Carlo simulation methods to assess the inflation of the type I error rate when testing the statistical significance of a risk factor after adjusting for a continuous confounding variable that has been divided into categories. We found that the inflation of the type I error rate increases with increasing sample size, as the correlation between the risk factor and the confounding variable increases, and with a decrease in the number of categories into which the confounder is divided. Even when the confounder is divided in a five-level categorical variable, the inflation of the type I error rate remained high when both the sample size and the correlation between the risk factor and the confounder were high.     

Time-to-event analysis with treatment arm selection at interim

This paper discusses the application of an adaptive design for treatment arm selection in an oncology trial, with survival as the primary endpoint and disease progression as a key secondary endpoint. We carried out treatment arm selection at an interim analysis by using Bayesian predictive power combining evidence from the two endpoints. At the final analysis, we carried out a frequentist statistical test of efficacy on the survival endpoint. We investigated several approaches (Bonferroni approach, 'Dunnett-like' approach, a conditional error function approach and a combination p-value approach) with respect to their power and the precise conditions under which type I error control is attained.     

Fallback tests for co‐primary endpoints

When efficacy of a treatment is measured by co‐primary endpoints, efficacy is claimed only if for each endpoint an individual statistical test is significant at level α. While such a strategy controls the family‐wise type I error rate (FWER), it is often strictly conservative and allows for no inference if not all null hypotheses can be rejected. In this paper, we investigate fallback tests, which are defined as uniform improvements of the classical test for co‐primary endpoints. They reject whenever the classical test rejects but allow for inference also in settings where only a subset of endpoints show a significant effect. Similarly to the fallback tests for hierarchical testing procedures, these fallback tests for co‐primary endpoints allow one to continue testing even if the primary objective of the trial was not met. We propose examples of fallback tests for two and three co‐primary endpoints that control the FWER in the strong sense under the assumption of multivariate normal test statistics with arbitrary correlation matrix and investigate their power in a simulation study. The fallback procedures for co‐primary endpoints are illustrated with a clinical trial in a rare disease and a diagnostic trial. © 2016 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.     

An efficient method for accommodating potentially underpowered primary endpoints

If a trial is adequately powered for two clinically important endpoints, A and B, each of which can fully characterize a treatment benefit to support a regulatory claim by itself, then both endpoints are usually labeled primary, and the trial is deemed positive if either endpoint is statistically significant after a multiplicity adjustment. However, if only A is adequately powered, then should B be designated a secondary endpoint, or should it be retained in the primary family despite being (potentially) underpowered? The former option can lead to a negative trial if A is not statistically significant, no matter how positive the results are for B, since no familywise type I error rate (FWER) is allocated to B, while the latter can reduce the likelihood of a positive trial if an inefficient multiplicity adjustment is used. We underscore this contemporary problem with real examples and offer a novel and intuitively appealing solution for accommodating clinically important but potentially underpowered endpoint(s) in the primary family. In our proposal, for the above scenario with two endpoints, A is tested at a prespecified level α₁=α?ε (e.g. ε=0.01 when α=0.05), and B at an ‘adaptive’ level α₂ (?α) calculated using a prespecified non‐increasing function of the p‐value for A. Our method controls the FWER at level α and can notably increase the probability of achieving a positive trial compared with a fixed prospective alpha allocation scheme (Control. Clin. Trials 2000; 20 :40–49), and with Hochberg's method applied to the family of primary endpoints. Importantly, our proposal enables strong results for potentially underpowered primary endpoint(s) to be interpreted in a conclusive rather than exploratory light. Copyright © 2008 John Wiley & Sons, Ltd.