Operating characteristics of sample size re-estimation with futility stopping based on conditional power

Various methods have been described for re-estimating the final sample size in a clinical trial based on an interim assessment of the treatment effect. Many re-weight the observations after re-sizing so as to control the pursuant inflation in the type I error probability alpha. Lan and Trost (Estimation of parameters and sample size re-estimation. Proceedings of the American Statistical Association Biopharmaceutical Section 1997; 48-51) proposed a simple procedure based on conditional power calculated under the current trend in the data (CPT). The study is terminated for futility if CPT < or = CL, continued unchanged if CPT > or = CU, or re-sized by a factor m to yield CPT = CU if CL < CPT < CU, where CL and CU are pre-specified probability levels. The overall level alpha can be preserved since the reduction due to stopping for futility can balance the inflation due to sample size re-estimation, thus permitting any form of final analysis with no re-weighting. Herein the statistical properties of this approach are described including an evaluation of the probabilities of stopping for futility or re-sizing, the distribution of the re-sizing factor m, and the unconditional type I and II error probabilities alpha and beta. Since futility stopping does not allow a type I error but commits a type II error, then as the probability of stopping for futility increases, alpha decreases and beta increases. An iterative procedure is described for choice of the critical test value and the futility stopping boundary so as to ensure that specified alpha and beta are obtained. However, inflation in beta is controlled by reducing the probability of futility stopping, that in turn dramatically increases the possible re-sizing factor m. The procedure is also generalized to limit the maximum sample size inflation factor, such as at m max = 4. However, doing so then allows for a non-trivial fraction of studies to be re-sized at this level that still have low conditional power. These properties also apply to other methods for sample size re-estimation with a provision for stopping for futility. Sample size re-estimation procedures should be used with caution and the impact on the overall type II error probability should be assessed.     

Increasing the sample size during clinical trials with t-distributed test statistics without inflating the type I error rate

In clinical trials with t-distributed test statistics the required sample size depends on the unknown variance. Taking estimates from previous studies often leads to a misspecification of the true value of the variance. Hence, re-estimation of the variance based on the collected data and re-calculation of the required sample size is attractive. We present a flexible method for extensions of fixed sample or group-sequential trials with t-distributed test statistics. The method can be applied at any time during the course of the trial and does not require the necessity to pre-specify a sample size re-calculation rule. All available information can be used to determine the new sample size. The advantage of our method when compared with other adaptive methods is maintenance of the efficient t-test design when no extensions are actually made. We show that the type I error rate is preserved.     

Information‐based sample size re‐estimation in group sequential design for longitudinal trials

Group sequential design has become more popular in clinical trials because it allows for trials to stop early for futility or efficacy to save time and resources. However, this approach is less well‐known for longitudinal analysis. We have observed repeated cases of studies with longitudinal data where there is an interest in early stopping for a lack of treatment effect or in adapting sample size to correct for inappropriate variance assumptions. We propose an information‐based group sequential design as a method to deal with both of these issues. Updating the sample size at each interim analysis makes it possible to maintain the target power while controlling the type I error rate. We will illustrate our strategy with examples and simulations and compare the results with those obtained using fixed design and group sequential design without sample size re‐estimation. Copyright © 2014 John Wiley & Sons, Ltd.     

Modified Haybittle-Peto group sequential designs for testing superiority and non-inferiority hypotheses in clinical trials

In designing an active controlled clinical trial, one sometimes has to choose between a superiority objective (to demonstrate that a new treatment is more effective than an active control therapy) and a non-inferiority objective (to demonstrate that it is no worse than the active control within some pre-specified non-inferiority margin). It is often difficult to decide which study objective should be undertaken at the planning stage when one does not have actual data on the comparative advantage of the new treatment. By making use of recent advances in the theory of efficient group sequential tests, we show how this difficulty can be resolved by a flexible group sequential design that can adaptively choose between the superiority and non-inferiority objectives during interim analyses. While maintaining the type I error probability at a pre-specified level, the proposed test is shown to have power advantage and/or sample size saving over fixed sample size tests for either only superiority or non-inferiority, and over other group sequential designs in the literature.     

Adaptive increase in sample size when interim results are promising: a practical guide with examples

This paper discusses the benefits and limitations of adaptive sample size re-estimation for phase 3 confirmatory clinical trials. Comparisons are made with more traditional fixed sample and group sequential designs. It is seen that the real benefit of the adaptive approach arises through the ability to invest sample size resources into the trial in stages. The trial starts with a small up-front sample size commitment. Additional sample size resources are committed to the trial only if promising results are obtained at an interim analysis. This strategy is shown through examples of actual trials, one in neurology and one in cardiology, to be more advantageous than the fixed sample or group sequential approaches in certain settings. A major factor that has generated controversy and inhibited more widespread use of these methods has been their reliance on non-standard tests and p-values for preserving the type-1 error. If, however, the sample size is only increased when interim results are promising, one can dispense with these non-standard methods of inference. Therefore, in the spirit of making adaptive increases in trial size more widely appealing and readily implementable we here define those promising circumstances in which a conventional final inference can be performed while preserving the overall type-1 error. Methodological, regulatory and operational issues are examined.     

Evaluating the adaptive performance of flexible sample size designs with treatment difference in an interval

In clinical trials, the study sample size is often chosen to provide specific power at a single point of a treatment difference. When this treatment difference is not close to the true one, the actual power of the trial can deviate from the specified power. To address this issue, we consider obtaining a flexible sample size design that provides sufficient power and has close to the 'ideal' sample size over possible values of the true treatment difference within an interval. A performance score is proposed to assess the overall performance of these flexible sample size designs. Its application to the determination of the best solution among considered candidate sample size designs is discussed and illustrated through computer simulations.     

Mixtures of prior distributions for predictive Bayesian sample size calculations in clinical trials

In this paper we propose a predictive Bayesian approach to sample size determination (SSD) and re‐estimation in clinical trials, in the presence of multiple sources of prior information. The method we suggest is based on the use of mixtures of prior distributions for the unknown quantity of interest, typically a treatment effect or an effects‐difference. Methodologies are developed using normal models with mixtures of conjugate priors. In particular we extend the SSD analysis of Gajewski and Mayo (Statist. Med. 2006; 25 :2554–2566) and the sample size re‐estimation technique of Wang (Biometrical J. 2006; 48 (5):1–13). Copyright © 2009 John Wiley & Sons, Ltd.     

Two‐stage design of clinical trials involving recurrent events

Mixed Poisson models are often used for the design of clinical trials involving recurrent events since they provide measures of treatment effect based on rate and mean functions and accommodate between individual heterogeneity in event rates. Planning studies based on these models can be challenging when there is a little information available on the population event rates, or the extent of heterogeneity characterized by the variance of individual‐specific random effects. We consider methods for adaptive two‐stage clinical trial design, which enable investigators to revise sample size estimates using data collected during the first phase of the study. We describe blinded procedures in which the group membership and treatment received by each individual are not revealed at the interim analysis stage, and a ‘partially blinded’ procedure in which group membership is revealed but not the treatment received by the groups. An EM algorithm is proposed for the interim analyses in both cases, and the performance is investigated through simulation. The work is motivated by the design of a study involving patients with immune thrombocytopenic purpura where the aim is to reduce bleeding episodes and an illustrative application is given using data from a cardiovascular trial. Copyright © 2009 John Wiley & Sons, Ltd.     

Blinded sample size re‐estimation in three‐arm trials with ‘gold standard’ design

In this article, we study blinded sample size re‐estimation in the ‘gold standard’ design with internal pilot study for normally distributed outcomes. The ‘gold standard’ design is a three‐arm clinical trial design that includes an active and a placebo control in addition to an experimental treatment. We focus on the absolute margin approach to hypothesis testing in three‐arm trials at which the non‐inferiority of the experimental treatment and the assay sensitivity are assessed by pairwise comparisons. We compare several blinded sample size re‐estimation procedures in a simulation study assessing operating characteristics including power and type I error. We find that sample size re‐estimation based on the popular one‐sample variance estimator results in overpowered trials. Moreover, sample size re‐estimation based on unbiased variance estimators such as the Xing–Ganju variance estimator results in underpowered trials, as it is expected because an overestimation of the variance and thus the sample size is in general required for the re‐estimation procedure to eventually meet the target power. To overcome this problem, we propose an inflation factor for the sample size re‐estimation with the Xing–Ganju variance estimator and show that this approach results in adequately powered trials. Because of favorable features of the Xing–Ganju variance estimator such as unbiasedness and a distribution independent of the group means, the inflation factor does not depend on the nuisance parameter and, therefore, can be calculated prior to a trial. Moreover, we prove that the sample size re‐estimation based on the Xing–Ganju variance estimator does not bias the effect estimate. Copyright © 2017 John Wiley & Sons, Ltd.     

Performance of adaptive sample size adjustment with respect to stopping criteria and time of interim analysis

The benefit of adjusting the sample size in clinical trials on the basis of treatment effects observed in interim analysis has been the subject of several recent papers. Different conclusions were drawn about the usefulness of this approach for gaining power or saving sample size, because of differences in trial design and setting. We examined the benefit of sample size adjustment in relation to trial design parameters such as 'time of interim analysis' and 'choice of stopping criteria'. We compared the adaptive weighted inverse normal method with classical group sequential methods for the most common and for optimal stopping criteria in early, half-time and late interim analyses. We found that reacting to interim data might significantly reduce average sample size in some situations, while classical approaches can out-perform the adaptive designs under other circumstances. We characterized these situations with respect to time of interim analysis and choice of stopping criteria.     

Robustness of sample size re-estimation procedure in clinical trials (arbitrary populations)

In clinical trials, one of the main questions that is being asked is how many additional observations, if any, are needed beyond those originally planned. In a two-treatment double-blind clinical experiment, one is interested in testing the null hypothesis of equality of the means against one-sided alternative when the common variance sigma2 is unknown. We wish to determine the required total sample size when the error probabilities alpha and beta are specified at a predetermined alternative. Shih provided a two-stage procedure which is an extension of Stein's one-sample procedure, assuming normal response. He estimates sigma2 by the method of maximum likelihood via the EM algorithm and carries out a simulation study in order to evaluate the effective level of significance and the power. The author proposed a closed-form estimator for sigma2 and showed analytically that the difference between the effective and nominal levels of significance is negligible and that the power exceeds 1-beta when the initial sample size is large. Here we consider responses from arbitrary distributions in which the mean and the variance are not functionally related and show that when the initial sample size is large, the conclusions drawn previously by the author still hold. The effective coverage probability of a fixed-width interval is also evaluated. Proofs of certain assertions are deferred to the Appendix.     

On the efficiency of adaptive sample size design

Adaptive sample size designs, including group sequential designs, have been used as alternatives to fixed sample size designs to achieve more robust statistical power and better trial efficiency. This work investigates the efficiency of adaptive sample size designs as compared to group sequential designs. We show that given a group sequential design, a uniformly more efficient adaptive sample size design based on the same maximum sample size and rejection boundary can be constructed. While maintaining stable statistical power at the required level, the expected sample size of the obtained adaptive sample size design is uniformly smaller than that of the group sequential design with respect to a range of the true treatment difference. The finding provides further insights into the efficiency of adaptive sample size designs and challenges the popular belief of better efficiency associated with group sequential designs. Good adaptive performance plus easy implementation and other desirable operational features make adaptive sample size designs more attractive and applicable to modern clinical trials.     

Robust Bayesian sample size determination in clinical trials

This article deals with determination of a sample size that guarantees the success of a trial. We follow a Bayesian approach and we say an experiment is successful if it yields a large posterior probability that an unknown parameter of interest (an unknown treatment effect or an effects-difference) is greater than a chosen threshold. In this context, a straightforward sample size criterion is to select the minimal number of observations so that the predictive probability of a successful trial is sufficiently large. In the paper we address the most typical criticism to Bayesian methods-their sensitivity to prior assumptions-by proposing a robust version of this sample size criterion. Specifically, instead of a single distribution, we consider a class of plausible priors for the parameter of interest. Robust sample sizes are then selected by looking at the predictive distribution of the lower bound of the posterior probability that the unknown parameter is greater than a chosen threshold. For their flexibility and mathematical tractability, we consider classes of epsilon-contamination priors. As specific applications we consider sample size determination for a Phase III trial.     

A new conditional performance score for the evaluation of adaptive group sequential designs with sample size recalculation

In standard clinical trial designs, the required sample size is fixed in the planning stage based on initial parameter assumptions. It is intuitive that the correct choice of the sample size is of major importance for an ethical justification of the trial. The required parameter assumptions should be based on previously published results from the literature. In clinical practice, however, historical data often do not exist or show highly variable results. Adaptive group sequential designs allow a sample size recalculation after a planned unblinded interim analysis in order to adjust the sample size during the ongoing trial. So far, there exist no unique standards to assess the performance of sample size recalculation rules. Single performance criteria commonly reported are given by the power and the average sample size; the variability of the recalculated sample size and the conditional power distribution are usually ignored. Therefore, the need for an adequate performance score combining these relevant performance criteria is evident. To judge the performance of an adaptive design, there exist two possible perspectives, which might also be combined: Either the global performance of the design can be addressed, which averages over all possible interim results, or the conditional performance is addressed, which focuses on the remaining performance conditional on a specific interim result. In this work, we give a compact overview of sample size recalculation rules and performance measures. Moreover, we propose a new conditional performance score and apply it to various standard recalculation rules by means of Monte-Carlo simulations.     

Efficient group sequential designs when there are several effect sizes under consideration

We consider the construction of efficient group sequential designs where the goal is a low expected sample size not only at the null hypothesis and the alternative (taken to be the minimal clinically meaningful effect size), but also at more optimistic anticipated effect sizes. Pre-specified Type I error rate and power requirements can be achieved both by standard group sequential tests and by more recently proposed adaptive procedures. We investigate four nested classes of designs: (A) group sequential tests with equal group sizes and stopping boundaries determined by a monomial error spending function (the 'rho-family'); (B) as A but the initial group size is allowed to be different from the others; (C) group sequential tests with arbitrary group sizes and arbitrary boundaries, fixed in advance; (D) adaptive tests-as C but at each analysis, future group sizes and critical values are updated depending on the current value of the test statistic. By examining the performance of optimal procedures within each class, we conclude that class B provides simple and efficient designs with efficiency close to that of the more complex designs of classes C and D. We provide tables and figures illustrating the performances of optimal designs within each class and defining the optimal procedures of classes A and B.     

A comparison of adaptive allocation rules for group-sequential binary response clinical trials

In clinical trials to compare two or more treatments with dichotomous responses, group-sequential designs may reduce the total number of patients involved in the trial and response-adaptive designs may result in fewer patients being assigned to the inferior treatments. In this paper, we combine group-sequential and response-adaptive designs, extending recent work on sample size re-estimation in trials to compare two treatments with normally distributed responses, to analogous binary response trials. We consider the use of two parameters of interest in the group-sequential design, the log odds ratio and the simple difference between the probabilities of success. In terms of the adaptive sampling rules, we study two urn models, the drop-the-loser rule and the randomized Pólya urn rule, and compare their properties with those of two sequential maximum likelihood estimation rules, which minimize the expected number of treatment failures. We investigate two ways in which adaptive urn designs can be used in conjunction with group-sequential designs. The first method updates the urn at each interim analysis and the second method continually updates the urn after each patient response, assuming immediate patient responses. Our simulation results show that the group-sequential design, which uses the drop-the-loser rule, applied fully sequentially, is the most effective method for reducing the expected number of treatment failures and the average sample number, whilst still maintaining the nominal error rates, over a range of success probabilities.     

A framework for two-stage adaptive procedures to simultaneously test non-inferiority and superiority

In clinical trials it is often desirable to test for non-inferiority and for superiority simultaneously. For such a situation a two-stage adaptive procedure may be advantageous to a conventional single-stage procedure because a two-stage adaptive procedure allows the design of stage II, including the main study objective and sample size, to depend on the outcome of stage I. We propose a framework for designing two-stage adaptive procedures with a possible switch of the primary study objectives at the end of stage I between non-inferiority and superiority. The framework permits control of the type I error rate and specification of the unconditional powers and maximum sample size for each of non-inferiority and superiority objectives. The actions at the end of stage I are predetermined as functions of the stage I observations, thus making specification of the unconditional powers possible. Based on the results at the end of stage I, the primary objective for stage II is chosen, and sample sizes and critical values for stage II are determined.     

Design and sample size estimation in clinical trials with clustered survival times as the primary endpoint

Many clinical trials involve the collection of data on the time to occurrence of the same type of multiple events within sample units, in which ordering of events is arbitrary and times are usually correlated. To design a clinical trial with this type of clustered survival times as the primary endpoint, estimating the number of subjects (sampling units) required for a given power to detect a specified treatment difference is an important issue. In this paper we derive a sample size formula for clustered survival data via Lee, Wei and Amato's marginal model. It can be easily used to plan a clinical trial in which clustered survival times are of primary interest. Simulation studies demonstrate that the formula works very well. We also discuss and compare cluster survival time design and single survival time design (for example, time to the first event) in different scenarios.     

Proper inference from Simon's two-stage designs

Simon's two-stage designs are very popular for phase II clinical trials. A literature review revealed that the inference procedures used with Simon's designs almost always ignore the actual sampling plan used. Reported P-values, point estimates and confidence intervals for the response rate are not usually adjusted for the design's adaptiveness. In addition, we found that the actual sample size for the second stage is often different from that planned. We present here a method for inferences using both the planned and the actual sample sizes. The conventional and the preferred inference procedures usually yield similar P-values and confidence intervals for the response rate. The conventional inference, however, may contradict the result of the corresponding hypothesis testing.     

Sample size determination in group‐sequential clinical trials with two co‐primary endpoints

We discuss sample size determination in group‐sequential designs with two endpoints as co‐primary. We derive the power and sample size within two decision‐making frameworks. One is to claim the test intervention's benefit relative to control when superiority is achieved for the two endpoints at the same interim timepoint of the trial. The other is when superiority is achieved for the two endpoints at any interim timepoint, not necessarily simultaneously. We evaluate the behaviors of sample size and power with varying design elements and provide a real example to illustrate the proposed sample size methods. In addition, we discuss sample size recalculation based on observed data and evaluate the impact on the power and Type I error rate. Copyright © 2014 John Wiley & Sons, Ltd.