Adaptive Dunnett tests for treatment selection

Clinical trials incorporating treatment selection at pre-specified interim analyses allow to integrate two clinical studies into a single, confirmatory study. In an adaptive interim analysis, treatment arms are selected based on interim data as well as external information. The specific selection rule does not need to be pre-specified in advance in order to control the multiple type I error rate. We propose an adaptive Dunnett test procedure based on the conditional error rate of the single-stage Dunnett test. The adaptive procedure uniformly improves the classical Dunnett test, which is shown to be strictly conservative if treatments are dropped at interim. The adaptive Dunnett test is compared in a simulation with the classical Dunnett test as well as with adaptive combination tests based on the closure principle. The method is illustrated with a real-data example.     

Unbiased estimation in seamless phase II/III trials with unequal treatment effect variances and hypothesis‐driven selection rules

Seamless phase II/III clinical trials offer an efficient way to select an experimental treatment and perform confirmatory analysis within a single trial. However, combining the data from both stages in the final analysis can induce bias into the estimates of treatment effects. Methods for bias adjustment developed thus far have made restrictive assumptions about the design and selection rules followed. In order to address these shortcomings, we apply recent methodological advances to derive the uniformly minimum variance conditionally unbiased estimator for two‐stage seamless phase II/III trials. Our framework allows for the precision of the treatment arm estimates to take arbitrary values, can be utilised for all treatments that are taken forward to phase III and is applicable when the decision to select or drop treatment arms is driven by a multiplicity‐adjusted hypothesis testing procedure. © 2016 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.     

Time-to-event analysis with treatment arm selection at interim

This paper discusses the application of an adaptive design for treatment arm selection in an oncology trial, with survival as the primary endpoint and disease progression as a key secondary endpoint. We carried out treatment arm selection at an interim analysis by using Bayesian predictive power combining evidence from the two endpoints. At the final analysis, we carried out a frequentist statistical test of efficacy on the survival endpoint. We investigated several approaches (Bonferroni approach, 'Dunnett-like' approach, a conditional error function approach and a combination p-value approach) with respect to their power and the precise conditions under which type I error control is attained.     

Adaptive designs for confirmatory clinical trials

Adaptive designs play an increasingly important role in clinical drug development. Such designs use accumulating data of an ongoing trial to decide how to modify design aspects without undermining the validity and integrity of the trial. Adaptive designs thus allow for a number of possible adaptations at midterm: Early stopping either for futility or success, sample size reassessment, change of population, etc. A particularly appealing application is the use of adaptive designs in combined phase II/III studies with treatment selection at interim. The expectation has arisen that carefully planned and conducted studies based on adaptive designs increase the efficiency of the drug development process by making better use of the observed data, thus leading to a higher information value per patient. In this paper we focus on adaptive designs for confirmatory clinical trials. We review the adaptive design methodology for a single null hypothesis and how to perform adaptive designs with multiple hypotheses using closed test procedures. We report the results of an extensive simulation study to evaluate the operational characteristics of the various methods. A case study and related numerical examples are used to illustrate the key results. In addition we provide a detailed discussion of current methods to calculate point estimates and confidence intervals for relevant parameters. Copyright © 2009 John Wiley & Sons, Ltd.     

Adaptive designs: looking for a needle in the haystack-a new challenge in medical research

The statistical principles of fully adaptive designs are outlined. The options of flexibility and the price to be paid in terms of statistical properties of the test procedures are discussed. It is stressed that controlled inference after major design modifications (changing hypotheses) will include a penalty: Intersections among all the hypotheses considered throughout the trial have to be rejected before testing individual hypotheses. Moreover, feasibility in terms of integrity and persuasiveness of the results achieved after adaptations based on unblinded data is considered as the crucial issue in practice. In the second part, sample size adaptive procedures are considered testing a large number of hypotheses under constraints on total sample size as in genetic studies. The advantage of sequential procedures is sketched for the example of two-stage designs with a pilot phase for screening promising hypotheses (markers) and controlling the false discovery rate. Finally, we turn to the clinical problem how to select markers and estimate a score from limited samples, e.g. for predicting the response to therapy of a future patient. The predictive ability of such scores will be rather poor when investigating a large number of hypotheses and truly large marker effects are lacking. An obvious dilemma will show up: More optimistic selection rules may be superior if in fact effective markers exist, but will produce more nuisance prediction if no effective markers exist compared with more cautious strategies, e.g. aiming at some control of type I error probabilities.     

Designing multi‐arm multi‐stage clinical trials using a risk–benefit criterion for treatment selection

Multi‐arm clinical trials that compare several active treatments to a common control have been proposed as an efficient means of making an informed decision about which of several treatments should be evaluated further in a confirmatory study. Additional efficiency is gained by incorporating interim analyses and, in particular, seamless Phase II/III designs have been the focus of recent research. Common to much of this work is the constraint that selection and formal testing should be based on a single efficacy endpoint, despite the fact that in practice, safety considerations will often play a central role in determining selection decisions. Here, we develop a multi‐arm multi‐stage design for a trial with an efficacy and safety endpoint. The safety endpoint is explicitly considered in the formulation of the problem, selection of experimental arm and hypothesis testing. The design extends group‐sequential ideas and considers the scenario where a minimal safety requirement is to be fulfilled and the treatment yielding the best combined safety and efficacy trade‐off satisfying this constraint is selected for further testing. The treatment with the best trade‐off is selected at the first interim analysis, while the whole trial is allowed to compose of J analyses. We show that the design controls the familywise error rate in the strong sense and illustrate the method through an example and simulation. We find that the design is robust to misspecification of the correlation between the endpoints and requires similar numbers of subjects to a trial based on efficacy alone for moderately correlated endpoints. © 2015 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.     

Adaptive increase in sample size when interim results are promising: a practical guide with examples

This paper discusses the benefits and limitations of adaptive sample size re-estimation for phase 3 confirmatory clinical trials. Comparisons are made with more traditional fixed sample and group sequential designs. It is seen that the real benefit of the adaptive approach arises through the ability to invest sample size resources into the trial in stages. The trial starts with a small up-front sample size commitment. Additional sample size resources are committed to the trial only if promising results are obtained at an interim analysis. This strategy is shown through examples of actual trials, one in neurology and one in cardiology, to be more advantageous than the fixed sample or group sequential approaches in certain settings. A major factor that has generated controversy and inhibited more widespread use of these methods has been their reliance on non-standard tests and p-values for preserving the type-1 error. If, however, the sample size is only increased when interim results are promising, one can dispense with these non-standard methods of inference. Therefore, in the spirit of making adaptive increases in trial size more widely appealing and readily implementable we here define those promising circumstances in which a conventional final inference can be performed while preserving the overall type-1 error. Methodological, regulatory and operational issues are examined.     

Interim treatment selection with a flexible selection margin in clinical trials

When several treatment arms are administered along with a control arm in a trial, dropping the non‐promising treatments at an early stage helps to save the resources and expedite the trial. In such adaptive designs with treatment selection, a common selection rule is to pick the most promising treatment, for example, the treatment with the numerically highest mean response, at the interim stage. However, with only a single treatment selected for final evaluation, this selection rule is often too inflexible. We modified this interim selection rule by introducing a flexible selection margin to judge the acceptable treatment difference. Another treatment could be selected at the interim stage in addition to the empirically best one if the differences of the observed treatment effect between them do not exceed this margin. We considered the study starting with two treatment arms and a control arm. We developed hypothesis testing procedures to assess the selected treatment(s) by taking into account the interim selection process. Compared with the one‐winner selection designs, the modified selection rule makes the design more flexible and practical. Copyright © 2012 John Wiley & Sons, Ltd.     

Adaptive extensions of a two-stage group sequential procedure for testing primary and secondary endpoints (II): sample size re-estimation

In this part II of the paper on adaptive extensions of a two‐stage group sequential procedure (GSP) for testing primary and secondary endpoints, we focus on the second stage sample size re‐estimation based on the first stage data. First, we show that if we use the Cui–Huang–Wang statistics at the second stage, then we can use the same primary and secondary boundaries as for the original procedure (without sample size re‐estimation) and still control the type I familywise error rate. This extends their result for the single endpoint case. We further show that the secondary boundary can be sharpened in this case by taking the unknown correlation coefficient ρ between the primary and secondary endpoints into account through the use of the confidence limit method proposed in part I of this paper. If we use the sufficient statistics instead of the CHW statistics, then we need to modify both the primary and secondary boundaries; otherwise, the error rate can get inflated. We show how to modify the boundaries of the original group sequential procedure to control the familywise error rate. We provide power comparisons between competing procedures. We illustrate the procedures with a clinical trial example. Copyright © 2012 John Wiley & Sons, Ltd.     

Multiplicity adjustment for multiple endpoints in clinical trials with multiple doses of an active treatment

Frequently, multiple doses of an active treatment and multiple endpoints are simultaneously considered in the designs of clinical trials. For these trials, traditional multiplicity adjustment procedures such as Bonferroni, Hochberg and Hommel procedures can be applied when treating the comparisons of different doses to the control on all endpoints at the same level. However, these approaches will not take into account the possible dose-response relationship on each endpoint, and therefore are less specific and may have lower power. To gain power, in this paper, we consider the problem as a two-dimensional multiplicity problem: one dimension concerns the multiple doses and the other dimension concerns the multiple endpoints. We propose procedures which consider the dose order to form the closure of the procedures and control the family-wise type I error rate in a strong sense. For this two-dimensional problem, numerical examples show that procedures proposed in this paper in general have higher power than the commonly used procedures (e.g. the regular Hochberg procedure) especially for comparing the higher dose to the control.     

Flexible selection of a single treatment incorporating short‐term endpoint information in a phase II/III clinical trial

Seamless phase II/III clinical trials in which an experimental treatment is selected at an interim analysis have been the focus of much recent research interest. Many of the methods proposed are based on the group sequential approach. This paper considers designs of this type in which the treatment selection can be based on short‐term endpoint information for more patients than have primary endpoint data available. We show that in such a case, the familywise type I error rate may be inflated if previously proposed group sequential methods are used and the treatment selection rule is not specified in advance. A method is proposed to avoid this inflation by considering the treatment selection that maximises the conditional error given the data available at the interim analysis. A simulation study is reported that illustrates the type I error rate inflation and compares the power of the new approach with two other methods: a combination testing approach and a group sequential method that does not use the short‐term endpoint data, both of which also strongly control the type I error rate. The new method is also illustrated through application to a study in Alzheimer's disease. © 2015 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.     

Adaptive sampling in two‐phase designs: a biomarker study for progression in arthritis

Response‐dependent two‐phase designs are used increasingly often in epidemiological studies to ensure sampling strategies offer good statistical efficiency while working within resource constraints. Optimal response‐dependent two‐phase designs are difficult to implement, however, as they require specification of unknown parameters. We propose adaptive two‐phase designs that exploit information from an internal pilot study to approximate the optimal sampling scheme for an analysis based on mean score estimating equations. The frequency properties of estimators arising from this design are assessed through simulation, and they are shown to be similar to those from optimal designs. The design procedure is then illustrated through application to a motivating biomarker study in an ongoing rheumatology research program. Copyright © 2015 © 2015 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.     

Propensity scores used for analysis of cluster randomized trials with selection bias: a simulation study

Cluster randomized trials (CRTs) are often prone to selection bias despite randomization. Using a simulation study, we investigated the use of propensity score (PS) based methods in estimating treatment effects in CRTs with selection bias when the outcome is quantitative. Of four PS‐based methods (adjustment on PS, inverse weighting, stratification, and optimal full matching method), three successfully corrected the bias, as did an approach using classical multivariable regression. However, they showed poorer statistical efficiency than classical methods, with higher standard error for the treatment effect, and type I error much smaller than the 5% nominal level. Copyright © 2013 John Wiley & Sons, Ltd.     

Adaptive extensions of a two-stage group sequential procedure for testing primary and secondary endpoints (I): unknown correlation between the endpoints

In a previous paper we studied a two‐stage group sequential procedure (GSP) for testing primary and secondary endpoints where the primary endpoint serves as a gatekeeper for the secondary endpoint. We assumed a simple setup of a bivariate normal distribution for the two endpoints with the correlation coefficient ρ between them being either an unknown nuisance parameter or a known constant. Under the former assumption, we used the least favorable value of ρ = 1 to compute the critical boundaries of a conservative GSP. Under the latter assumption, we computed the critical boundaries of an exact GSP. However, neither assumption is very practical. The ρ = 1 assumption is too conservative resulting in loss of power, whereas the known ρ assumption is never true in practice. In this part I of a two‐part paper on adaptive extensions of this two‐stage procedure (part II deals with sample size re‐estimation), we propose an intermediate approach that uses the sample correlation coefficient r from the first‐stage data to adaptively adjust the secondary boundary after accounting for the sampling error in r via an upper confidence limit on ρ by using a method due to Berger and Boos. We show via simulation that this approach achieves 5–11% absolute secondary power gain for ρ ≤0.5. The preferred boundary combination in terms of high primary as well as secondary power is that of O'Brien and Fleming for the primary and of Pocock for the secondary. The proposed approach using this boundary combination achieves 72–84% relative secondary power gain (with respect to the exact GSP that assumes known ρ). We give a clinical trial example to illustrate the proposed procedure. Copyright © 2012 John Wiley & Sons, Ltd.     

Developing a Bayesian adaptive design for a phase I clinical trial: a case study for a novel HIV treatment

The design of phase I studies is often challenging, because of limited evidence to inform study protocols. Adaptive designs are now well established in cancer but much less so in other clinical areas. A phase I study to assess the safety, pharmacokinetic profile and antiretroviral efficacy of C34‐PEG₄‐Chol, a novel peptide fusion inhibitor for the treatment of HIV infection, has been set up with Medical Research Council funding. During the study workup, Bayesian adaptive designs based on the continual reassessment method were compared with a more standard rule‐based design, with the aim of choosing a design that would maximise the scientific information gained from the study. The process of specifying and evaluating the design options was time consuming and required the active involvement of all members of the trial's protocol development team. However, the effort was worthwhile as the originally proposed rule‐based design has been replaced by a more efficient Bayesian adaptive design. While the outcome to be modelled, design details and evaluation criteria are trial specific, the principles behind their selection are general. This case study illustrates the steps required to establish a design in a novel context. © 2016 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd     

Multiplicity in confirmatory clinical trials: a case study with discussion from a JSM panel

An invited panel session was conducted in the 2012 Joint Statistical Meetings, San Diego, California, USA, to stimulate the discussion on multiplicity issues in confirmatory clinical trials for drug development. A total of 11 expert panel members were invited and 9 participated. Prior to the session, a case study was previously provided to the panel members to facilitate the discussion, focusing on the key components of the study design and multiplicity. The Phase 3 development program for this new experimental treatment was based on a single randomized controlled trial alone. Each panelist was asked to clarify if he or she responded as if he or she were a pharmaceutical drug sponsor, an academic panelist or a health regulatory scientist. Copyright © 2015 John Wiley & Sons, Ltd.     

Assessing the impact of selection bias on test decisions in trials with a time‐to‐event outcome

If past treatment assignments are unmasked, selection bias may arise even in randomized controlled trials. The impact of such bias can be measured by considering the type I error probability. In case of a normally distributed outcome, there already exists a model accounting for selection bias that permits calculating the corresponding type I error probabilities. To model selection bias for trials with a time‐to‐event outcome, we introduce a new biasing policy for exponentially distributed data. Using this biasing policy, we derive an exact formula to compute type I error probabilities whenever an F‐test is performed and no observations are censored. Two exemplary settings, with and without random censoring, are considered in order to illustrate how our results can be applied to compare distinct randomization procedures with respect to their performance in the presence of selection bias. © 2017 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.     

Parsimonious conditional‐mean model selection with multiple covariates: an analysis of infant mortality in the USA

This paper proposes and evaluates an objective methodology to select a parsimonious conditional‐mean model when faced with multiple candidate predictor variables. The methodology attempts to fine‐tune a well‐established covariate screening method such as iterative sure independence screening with smoothly clipped absolute deviation penalty by using the following: (i) cross‐validated or bootstrap estimates of prediction error; (ii) an objective model comparison strategy; and (iii) multiple hypothesis testing. The methods are analytically and numerically shown to work well in the sense that the probability that the final model selected contains one or more unimportant variables is asymptotically bounded at a preselected level for arbitrary data‐generating distributions. This methodology is illustrated with a dataset consisting of birth certificate information and mortality records from year 2001 from the US Department of Health and Human Services on non‐Hispanic African American female and male infants. It is shown how the instantaneous daily mortality hazard can be modeled flexibly by allowing both the set of important predictors and their effect on the hazard to change arbitrarily thru time. Results indicate that once controlling for birth weight, no other variables on the birth certificate are significantly associated with mortality; furthermore, time and sex modify the birth weight/survival relationship, with the strongest association at earliest days and low birth weight female infants having a better survival experience than male counterparts. Copyright © 2013 John Wiley & Sons, Ltd.     

Conditioning on the propensity score can result in biased estimation of common measures of treatment effect: a Monte Carlo study

Propensity score methods are increasingly being used to estimate causal treatment effects in the medical literature. Conditioning on the propensity score results in unbiased estimation of the expected difference in observed responses to two treatments. The degree to which conditioning on the propensity score introduces bias into the estimation of the conditional odds ratio or conditional hazard ratio, which are frequently used as measures of treatment effect in observational studies, has not been extensively studied. We conducted Monte Carlo simulations to determine the degree to which propensity score matching, stratification on the quintiles of the propensity score, and covariate adjustment using the propensity score result in biased estimation of conditional odds ratios, hazard ratios, and rate ratios. We found that conditioning on the propensity score resulted in biased estimation of the true conditional odds ratio and the true conditional hazard ratio. In all scenarios examined, treatment effects were biased towards the null treatment effect. However, conditioning on the propensity score did not result in biased estimation of the true conditional rate ratio. In contrast, conventional regression methods allowed unbiased estimation of the true conditional treatment effect when all variables associated with the outcome were included in the regression model. The observed bias in propensity score methods is due to the fact that regression models allow one to estimate conditional treatment effects, whereas propensity score methods allow one to estimate marginal treatment effects. In several settings with non-linear treatment effects, marginal and conditional treatment effects do not coincide.     

Propensity score and difference-in-difference methods: a study of second-generation antidepressant use in patients with bipolar disorder

This article compared standard regression (logistic), propensity score weighting, propensity score matching, and difference-in-difference (DID) methods in determining the impact of second-generation antidepressant (AD) use on mania-related visits among adult patients with bipolar disorder. Using a large managed care claims database, a logistic regression was developed as a standard approach to predict the likelihood of having mania-related visits after receiving various types of treatments (AD monotherapy, mood stabilizer (MS) monotherapy, and AD-MS combination therapy) controlling for individual baseline characteristics. The propensity score method predicted the propensity to be with one treatment type versus another in the first-stage. Both weighting and greedy matching approaches were applied in the second-stage outcome model. For the DID method, a logistic regression was applied to predict the differential likelihood of having mania-related visits in post-baseline versus baseline periods on different treatments. Both full sample and propensity score-matched sample were applied for the DID method. Except DID with full sample, the results from all other methods suggested no higher likelihood of mania-related visits for second-generation AD-related therapies compared to MS monotherapy. We concluded that standard regression, propensity scoring, and DID methods may produce inconsistent outcomes in a logistic regression framework, when patient baseline characteristics are different between comparison groups and/or not all potential confounders can be correctly measured and fully controlled. Researchers need to be cautious of the basic assumptions and sensitivities of various methods before making a final conclusion. The DID method may be considered in outcome studies when pre-and-post data are available.