Efficient adaptive designs with mid-course sample size adjustment in clinical trials

Adaptive designs have been proposed for clinical trials in which the nuisance parameters or alternative of interest are unknown or likely to be misspecified before the trial. Although most previous works on adaptive designs and mid-course sample size re-estimation have focused on two-stage or group-sequential designs in the normal case, we consider here a new approach that involves at most three stages and is developed in the general framework of multiparameter exponential families. This approach not only maintains the prescribed type I error probability but also provides a simple but asymptotically efficient sequential test whose finite-sample performance, measured in terms of the expected sample size and power functions, is shown to be comparable to the optimal sequential design, determined by dynamic programming, in the simplified normal mean case with known variance and prespecified alternative, and superior to the existing two-stage designs and also to adaptive group-sequential designs when the alternative or nuisance parameters are unknown or misspecified.     

Optimal two-stage designs for clinical trials based on safety and efficacy

In clinical trials designed to evaluate treatment efficacy, it is common practice to terminate a treatment arm in which the observed rate of an adverse event is unacceptably high. This practice may be formalized by a group-sequential test based on a multivariate outcome including both adverse and efficacy events. Recently, Thall and Cheng proposed a family of tests for randomized trials of an experimental treatment versus a standard where patient outcome is bivariate with entries characterizing efficacy and safety. The test is motivated by the idea that clinically meaningful improvements over the standard may be characterized by a two-dimensional parameter quantifying trade-offs between efficacy and safety. We provide optimal two-stage designs based on this test that minimize either the mean sample size under the null hypothesis of no treatment difference, or the maximum sample size if the trial continues to a second stage. A more general group-sequential version of the design also is described, an illustration is provided, and application to the special case of single-arm phase II trials is discussed.     

Mid-trial design reviews for sequential clinical trials

When sequential clinical trials are conducted by plotting a statistic measuring treatment difference against another measuring information, power is guaranteed regardless of nuisance parameters. However, values need to be assigned to nuisance parameters in order to gain an impression of the sample size distribution. Each interim analysis provides an opportunity to re-evaluate the relationship between sample size and information. In this paper we discuss such mid-trial design reviews. In the special cases of trials with a relatively short recruitment phase followed by a longer period of follow-up, and of normally distributed responses, mid-trial design reviews are particularly important. Examples are given of the various situations considered, and extensive simulations are reported demonstrating the validity of the review procedure in the case of normally distributed responses.     

Implications of overviews of randomized trials

Many randomized trials are of insufficient sample size to detect with adequate power the small to moderate effects that are most likely to occur. As a result, a single such trial can produce a null finding that is, in fact, uninformative, but none the less is misinterpreted as demonstrating no effect. An overview considers all available trials and can increase the statistical power to detect an effect if present. Thus overviews can provide perhaps the most precise estimate of the magnitude of a treatment effect based on existing data. This may have implications for the formulation of public policy but certainly should influence the conduct and planning of randomized trials. Public policy may be influenced in circumstances where further trials are unlikely to be conducted. Overviews can also provide guidance as to whether changes in protocols of ongoing studies are recommended as a result of new evidence. Perhaps most importantly, overviews can provide information about whether additional trials are warranted, and, if so, the sample size that would be required to answer the research question definitively.     

Internal pilot design for balanced repeated measures

Repeated measures are common in clinical trials and epidemiological studies. Designing studies with repeated measures requires reasonably accurate specifications of the variances and correlations to select an appropriate sample size. Underspecifying the variances leads to a sample size that is inadequate to detect a meaningful scientific difference, while overspecifying the variances results in an unnecessarily large sample size. Both lead to wasting resources and placing study participants in unwarranted risk. An internal pilot design allows sample size recalculation based on estimates of the nuisance parameters in the covariance matrix. We provide the theoretical results that account for the stochastic nature of the final sample size in a common class of linear mixed models. The results are useful for designing studies with repeated measures and balanced design. Simulations examine the impact of misspecification of the covariance matrix and demonstrate the accuracy of the approximations in controlling the type I error rate and achieving the target power. The proposed methods are applied to a longitudinal study assessing early antiretroviral therapy for youth living with HIV.     

Sample size re-estimation in cluster randomization trials

Cluster randomization trials in which families are the unit of allocation are commonly adopted for the evaluation of disease prevention interventions. Sample size estimation for cluster randomization trials depends on parameters that quantify the variability within and between clusters and the variability in cluster size. Accurate advance estimates of these nuisance parameters may be difficult to obtain and misspecification may lead to an underpowered study. Since families are typically recruited over time, we propose using a portion of the data to estimate the nuisance parameters and to re-estimate sample size based on the estimates. This extends the standard internal pilot study methods to the setting of cluster randomization trials. The effect of this design on the power, significance level and sample size is analysed via simulation and is shown to provide a flexible and practical approach to cluster randomization trials.     

A comparison of adaptive allocation rules for group-sequential binary response clinical trials

In clinical trials to compare two or more treatments with dichotomous responses, group-sequential designs may reduce the total number of patients involved in the trial and response-adaptive designs may result in fewer patients being assigned to the inferior treatments. In this paper, we combine group-sequential and response-adaptive designs, extending recent work on sample size re-estimation in trials to compare two treatments with normally distributed responses, to analogous binary response trials. We consider the use of two parameters of interest in the group-sequential design, the log odds ratio and the simple difference between the probabilities of success. In terms of the adaptive sampling rules, we study two urn models, the drop-the-loser rule and the randomized Pólya urn rule, and compare their properties with those of two sequential maximum likelihood estimation rules, which minimize the expected number of treatment failures. We investigate two ways in which adaptive urn designs can be used in conjunction with group-sequential designs. The first method updates the urn at each interim analysis and the second method continually updates the urn after each patient response, assuming immediate patient responses. Our simulation results show that the group-sequential design, which uses the drop-the-loser rule, applied fully sequentially, is the most effective method for reducing the expected number of treatment failures and the average sample number, whilst still maintaining the nominal error rates, over a range of success probabilities.     

Inadequate statistical power to detect clinically significant differences in adverse event rates in randomized controlled trials

ObjectiveTo determine the statistical power to detect potentially clinically significant differences in serious adverse events between drug therapies reported in a sample of randomized controlled trials (RCTs).Study Design and SettingSystematic review of RCTs with positive efficacy endpoint and at least a twofold difference in the proportion of patients with serious adverse events between treatment groups from six major journals. The power of each study to detect statistically significant differences in serious adverse events was calculated.ResultsOf the six included trials, all performed statistical analysis on adverse events without disclosure of the statistical power for detecting the reported differences between groups. The power of each study to detect the reported differences in adverse events was calculated and yielded values ranging from 0.07 to 0.37 among trials with non–statistically significant differences.ConclusionStatistical testing for differences in the proportion of patients experiencing an adverse event is common in RCTs; non–statistically significant differences are associated with low statistical power. A high probability of type II error may lead to erroneous clinical inference resulting in harm. The statistical power for nonsignificant tests should be considered in the interpretation of results.     

Sample size re-estimation in group-sequential response-adaptive clinical trials

In clinical trials where the variances of the response variables are unknown, in accurate estimates of these can affect the type II error rate considerably. More accurate estimates of the variances may be obtained by taking a look at the data available part way through the trial and re-calculating the required sample size based on these new estimates. The main impetus for sample size re-estimation came from a two-stage procedure developed by Stein in 1945 and the literature is now replete with variations on this approach. In this paper, existing sample size re-estimation methods for both fixed sample and sequential clinical trial models will be reviewed. These will then be extended for use in group-sequential response-adaptive designs. In particular, a test for a recently developed group-sequential response-adaptive design, which compares two treatments with immediate normally distributed responses and unknown variances, is presented based on a modified version of Stein's test. The principal modifications involve updating the required sample size at each interim analysis and calculating the test statistic based on the current estimates of the variances. Hence, all the available information is used at each stage. Simulation is used to assess to what extent the updating of the required sample size at each interim analysis in the new test helps to attain the nominal error rates. The test is compared to modified versions of a simple test and a Stein-type group sequential t-test studied in the recent literature. These tests calculate the required sample sizes based on less accurate estimates of the variances. The type I error rate is close to the nominal value and the power is more accurately maintained in the new test.     

Individual allocation had an advantage over cluster randomization in statistical efficiency in some circumstances.

OBJECTIVE: To investigate the trade off between performing an individual randomized trial with a Complier Average Causal Effect analysis and accepting the fact that there will be some contamination, with a cluster randomized trial and the subsequent effect on the sample size and power of the trial. STUDY DESIGN AND SETTING: Monte Carlo simulations were undertaken to generate trial data where there was some contamination of control participants. The trials were simulated so the null hypothesis was false. Assessments were made of whether a type II error had been committed (i.e., whether the false null had not been rejected). RESULTS: As contamination increases, the power of the study to detect a true difference between the two groups declines. Using a Complier Average Causal Effect approach, unless anticipated contamination exceeds 30%, retains a sample size advantage over the cluster randomized design despite a relatively small cluster (i.e., 10) and reasonably small ICC (i.e., 0.04). CONCLUSION: If contamination can be measured precisely or estimated then in some circumstances individual allocation has an advantage over cluster randomization in statistical efficiency.     

Blinded sample size reestimation with count data: Methods and applications in multiple sclerosis

Sample size estimation in clinical trials depends critically on nuisance parameters, such as variances or overall event rates, which have to be guessed or estimated from previous studies in the planning phase of a trial. Blinded sample size reestimation estimates these nuisance parameters based on blinded data from the ongoing trial, and allows to adjust the sample size based on the acquired information. In the present paper, this methodology is developed for clinical trials with count data as the primary endpoint. In multiple sclerosis such endpoints are commonly used in phase 2 trials (lesion counts in magnetic resonance imaging (MRI)) and phase 3 trials (relapse counts). Sample size adjustment formulas are presented for both Poisson‐distributed data and for overdispersed Poisson‐distributed data. The latter arise from sometimes considerable between‐patient heterogeneity, which can be observed in particular in MRI lesion counts. The operation characteristics of the procedure are evaluated by simulations and recommendations on how to choose the size of the internal pilot study are given. The results suggest that blinded sample size reestimation for count data maintains the required power without an increase in the type I error. Copyright © 2010 John Wiley & Sons, Ltd.     

Powering our way to the elusive side effect: a composite outcome 'basket' of predefined designated endpoints in each organ system should be included in all controlled trials

OBJECTIVE: To review the methodological insights gained from the CLASS and VIGOR coxib trials and make suggestions for primary safety outcomes in future trials. STUDY DESIGN AND SETTING: We reviewed the methodology and findings of recent coxib trials and similar studies that have produced considerable controversy. RESULTS: There were statistical power problems with the primary and unexpected endpoints in both coxib trials. One approach for decreasing sample size requirements is using composite outcomes, wherein different clinically relevant endpoints are combined. By virtue of increasing the event rate fewer patients are required to detect a relative treatment effect of 50%. CONCLUSIONS: The recent withdrawal from the market of rofecoxib took years; the approach proposed here could shorten this time. It is important to revisit the comprehensiveness of the ability to detect important outcomes. Composite outcomes can be more varied in content, including mortality and morbidity. The proposed basket would consist of predefined designated endpoints in each organ system that could be associated with the population one is treating, and not just expected to be related to treatment. The goal is to anticipate and have a reasonable chance of detecting the unexpected side effect.     

Longitudinal Assessment Design and Statistical Power for Detecting an Intervention Impact

In evaluating randomized control trials (RCTs), statistical power analyses are necessary to choose a sample size which strikes the balance between an insufficient and an excessive design, with the latter leading to misspent resources. With the growing popularity of using longitudinal data to evaluate RCTs, statistical power calculations have become more complex. Specifically, with repeated measures, the number and frequency of measurements per person additionally influence statistical power by determining the precision with which intra-individual change can be measured as well as the reliability with which inter-individual differences in change can be assessed. The application of growth mixture models has shown that the impact of universal interventions is often concentrated among a small group of individuals at the highest level of risk. General sample size calculations were consequently not sufficient to determine whether statistical power is adequate to detect the desired effect. Currently, little guidance exists to recommend a sufficient assessment design to evaluating intervention impact. To this end, Monte Carlo simulations are conducted to assess the statistical power and precision when manipulating study duration and assessment frequency. Estimates were extracted from a published evaluation of the proximal of the Good Behavior Game (GBG) on the developmental course of aggressive behavior. Results indicated that the number of time points and the frequency of assessments influence statistical power and precision. Recommendations for the assessment design of longitudinal studies are discussed.     

Sample size re-estimation for clinical trials with longitudinal negative binomial counts including time trends

In some diseases, such as multiple sclerosis, lesion counts obtained from magnetic resonance imaging (MRI) are used as markers of disease progression. This leads to longitudinal, and typically overdispersed, count data outcomes in clinical trials. Models for such data invariably include a number of nuisance parameters, which can be difficult to specify at the planning stage, leading to considerable uncertainty in sample size specification. Consequently, blinded sample size re-estimation procedures are used, allowing for an adjustment of the sample size within an ongoing trial by estimating relevant nuisance parameters at an interim point, without compromising trial integrity. To date, the methods available for re-estimation have required an assumption that the mean count is time-constant within patients. We propose a new modeling approach that maintains the advantages of established procedures but allows for general underlying and treatment-specific time trends in the mean response. A simulation study is conducted to assess the effectiveness of blinded sample size re-estimation methods over fixed designs. Sample sizes attained through blinded sample size re-estimation procedures are shown to maintain the desired study power without inflating the Type I error rate and the procedure is demonstrated on MRI data from a recent study in multiple sclerosis.     

On the analysis of very small samples of Gaussian repeated measurements: an alternative approach

The analysis of very small samples of Gaussian repeated measurements can be challenging. First, due to a very small number of independent subjects contributing outcomes over time, statistical power can be quite small. Second, nuisance covariance parameters must be appropriately accounted for in the analysis in order to maintain the nominal test size. However, available statistical strategies that ensure valid statistical inference may lack power, whereas more powerful methods may have the potential for inflated test sizes. Therefore, we explore an alternative approach to the analysis of very small samples of Gaussian repeated measurements, with the goal of maintaining valid inference while also improving statistical power relative to other valid methods. This approach uses generalized estimating equations with a bias‐corrected empirical covariance matrix that accounts for all small‐sample aspects of nuisance correlation parameter estimation in order to maintain valid inference. Furthermore, the approach utilizes correlation selection strategies with the goal of choosing the working structure that will result in the greatest power. In our study, we show that when accurate modeling of the nuisance correlation structure impacts the efficiency of regression parameter estimation, this method can improve power relative to existing methods that yield valid inference. Copyright © 2017 John Wiley & Sons, Ltd.     

Evaluating the adaptive performance of flexible sample size designs with treatment difference in an interval

In clinical trials, the study sample size is often chosen to provide specific power at a single point of a treatment difference. When this treatment difference is not close to the true one, the actual power of the trial can deviate from the specified power. To address this issue, we consider obtaining a flexible sample size design that provides sufficient power and has close to the 'ideal' sample size over possible values of the true treatment difference within an interval. A performance score is proposed to assess the overall performance of these flexible sample size designs. Its application to the determination of the best solution among considered candidate sample size designs is discussed and illustrated through computer simulations.     

Sample size recalculation using conditional power

The sample size required to achieve a given power at a prespecified absolute difference in mean response may depend on one or more nuisance parameters, which are usually unknown. Proposed methods for using an internal pilot to recalculate the sample size using estimates of these parameters have been well studied. Most of these methods ignore the fact that data on the parameter of interest from within this internal pilot will contribute towards the value of the final test statistic. We propose a method which involves recalculating the target sample size by computing the number of further observations required to maintain the probability of rejecting the null hypothesis at the end of the study under the prespecified absolute difference in mean response conditional on the data observed so far. We do this within the framework of a two-group error-spending sequential test, modified so as to prevent inflation of the type I error rate.     

Power analyses for correlations from clustered study designs

Power analysis constitutes an important component of modern clinical trials and research studies. Although a variety of methods and software packages are available, almost all of them are focused on regression models, with little attention paid to correlation analysis. However, the latter is arguably a simpler and more appropriate approach for modelling concurrent events, especially in psychosocial research. In this paper, we discuss power and sample size estimation for correlation analysis arising from clustered study designs. Our approach is based on the asymptotic distribution of correlated Pearson-type estimates. Although this asymptotic distribution is easy to use in data analysis, the presence of a large number of parameters creates a major problem for power analysis due to the lack of real data to estimate them. By introducing a surrogacy-type assumption, we show that all nuisance parameters can be eliminated, making it possible to perform power analysis based only on the parameters of interest. Simulation results suggest that power and sample size estimates obtained under the proposed approach are robust to this assumption.     

Blinded sample size re‐estimation in three‐arm trials with ‘gold standard’ design

In this article, we study blinded sample size re‐estimation in the ‘gold standard’ design with internal pilot study for normally distributed outcomes. The ‘gold standard’ design is a three‐arm clinical trial design that includes an active and a placebo control in addition to an experimental treatment. We focus on the absolute margin approach to hypothesis testing in three‐arm trials at which the non‐inferiority of the experimental treatment and the assay sensitivity are assessed by pairwise comparisons. We compare several blinded sample size re‐estimation procedures in a simulation study assessing operating characteristics including power and type I error. We find that sample size re‐estimation based on the popular one‐sample variance estimator results in overpowered trials. Moreover, sample size re‐estimation based on unbiased variance estimators such as the Xing–Ganju variance estimator results in underpowered trials, as it is expected because an overestimation of the variance and thus the sample size is in general required for the re‐estimation procedure to eventually meet the target power. To overcome this problem, we propose an inflation factor for the sample size re‐estimation with the Xing–Ganju variance estimator and show that this approach results in adequately powered trials. Because of favorable features of the Xing–Ganju variance estimator such as unbiasedness and a distribution independent of the group means, the inflation factor does not depend on the nuisance parameter and, therefore, can be calculated prior to a trial. Moreover, we prove that the sample size re‐estimation based on the Xing–Ganju variance estimator does not bias the effect estimate. Copyright © 2017 John Wiley & Sons, Ltd.     

Accounting for cluster randomization: a review of primary prevention trials, 1990 through 1993.

OBJECTIVES. This methodological review aims to determine the extent to which design and analysis aspects of cluster randomization have been appropriately dealt with in reports of primary prevention trials. METHODS. All reports of primary prevention trials using cluster randomization that were published from 1990 to 1993 in the American Journal of Public Health and Preventive Medicine were identified. Each article was examined to determine whether cluster randomization was taken into account in the design and statistical analysis. RESULTS. Of the 21 articles, only 4 (19%) included sample size calculations or discussions of power that allowed for clustering, while 12 (57%) took clustering into account in the statistical analysis. CONCLUSIONS. Design and analysis issues associated with cluster randomization are not recognized widely enough. Reports of cluster randomized trials should include sample size calculations and statistical analyses that take clustering into account, estimates of design effects to help others planning trials, and a table showing the baseline distribution of important characteristics by intervention group, including the number of clusters and average cluster size for each group.