Improving the reporting of non-inferiority trials by incorporating non-efficacy benefits: not all non-inferiority trials are created equal

Non-inferiority trials are used to test if a novel intervention is not worse than a standard treatment by more than a prespecified amount, the non-inferiority margin (ΔNI). The ΔNI indicates the amount of efficacy loss in the primary outcome that is acceptable in exchange for non-efficacy benefits in other outcomes. However, non-inferiority designs are sometimes used when non-efficacy benefits are absent. Without non-efficacy benefits, loss in efficacy cannot be easily justified. Further, non-efficacy benefits are scarcely defined or considered by trialists when determining the magnitude of and providing justification for the non-inferiority margin. This is problematic as the importance of a treatment’s non-efficacy benefits are critical to understanding the results of a non-inferiority study. Here we propose the routine reporting in non-inferiority trial protocols and publications of non-efficacy benefits of the novel intervention along with the reporting of non-inferiority margins and their justification. The justification should include the specific trade-off between the accepted loss in efficacy (ΔNI) and the non-efficacy benefits of the novel treatment and should describe whether patients and other relevant stakeholders were involved in the definition of the ΔNI.

     

Group sequential designs for three‐arm ‘gold standard’ non‐inferiority trials with fixed margin

In the recent years there have been numerous publications on the design and the analysis of three‐arm ‘gold standard’ noninferiority trials. Whenever feasible, regulatory authorities recommend the use of such three‐arm designs including a test treatment, an active control, and a placebo. Nevertheless, it is desirable in many respects, for example, ethical reasons, to keep the placebo group size as small as possible. We first give a short overview on the fixed sample size design of a three‐arm noninferiority trial with normally distributed outcomes and a fixed noninferiority margin. An optimal single stage design is derived that should serve as a benchmark for the group sequential designs proposed in the main part of this work. It turns out, that the number of patients allocated to placebo is substantially low for the optimal design. Subsequently, approaches for group sequential designs aiming to further reduce the expected sample sizes are presented. By means of choosing different rejection boundaries for the respective null hypotheses, we obtain designs with quite different operating characteristics. We illustrate the approaches via numerical calculations and a comparison with the optimal single stage design. Furthermore, we derive approximately optimal boundaries for different goals, for example, to reduce the overall average sample size. The results show that the implementation of a group sequential design further improves the optimal single stage design. Besides cost and time savings, the possible early termination of the placebo arm is a key advantage that could help to overcome ethical concerns. Copyright © 2013 John Wiley & Sons, Ltd.     

Imbalanced randomization in clinical trials

Randomization is a common technique used in clinical trials to eliminate potential bias and confounders in a patient population. Equal allocation to treatment groups is the standard due to its optimal efficiency in many cases. However, in certain scenarios, unequal allocation can improve efficiency. In superiority trials with more than two groups, the optimal randomization is not always a balanced randomization. In noninferiority (NI) trials, additive margin with equal variance is the only instance with balanced randomization. Optimal randomization for NI trials can be far from 1:1 and can greatly improve efficiency, a fact which is commonly overlooked. A tool for sample size calculation for NI trials with additive or multiplicative margin with normal, binomial, or Poisson distribution is available at http://www.statlab.wisc.edu/shiny/SSNI/ .     

Likelihood ratio and a Bayesian approach were superior to standard noninferiority analysis when the noninferiority margin varied with the control event rate

OBJECTIVE: To present and compare three statistical approaches for analyzing a noninferiority trial when the noninferiority margin depends on the control event rate. STUDY DESIGN AND SETTING: In noninferiority trials with a binary outcome, the noninferiority margin is often defined as a fixed delta, the largest clinically acceptable difference in event rates between treatment groups. An alternative and more flexible approach is to allow delta to vary according to the true event rate in the control group. The appropriate statistical method for evaluating noninferiority with a variable noninferiority margin is not apparent. Three statistical approaches are proposed and compared: an observed event rate (OER) approach based on equating the true control rate to the observed rate, a Bayesian approach, and a likelihood ratio test. RESULTS AND CONCLUSIONS: Simulations studies indicate that the proportion of trials in which noninferiority was erroneously demonstrated was higher for the OER approach than with the Bayesian and likelihood ratio approaches. In some cases, the Type I error rate exceeded 10% for the OER approach. The OER approach is not recommended for the analysis of noninferiority trials with a variable margin of equivalence. The Bayesian and likelihood ratio methods yielded better operating characteristics.     

Current issues in non-inferiority trials

Non-inferiority (NI) trials enable a direct comparison of the relative benefit-to-risk profiles of an experimental intervention and a standard-of-care regimen. When the standard has clinical efficacy of substantial magnitude that is precisely estimated ideally using data from multiple adequate and well-controlled trials, with such estimates being relevant to the setting of the NI trial, then the NI trial can provide the scientific and regulatory evidence required to reliably assess the efficacy of the new intervention. In clinical practice, considerable uncertainty remains regarding when such trials should be conducted, how they should be designed, what standards for quality of trial conduct must be achieved, and how results should be interpreted. Recent examples will be considered to provide important insights and to highlight some of the challenges that remain to be adequately addressed regarding the use of the NI approach for the evaluation of new interventions. 'Imputed placebo' and 'margin'-based approaches to NI trial design will be considered, as well as the risk of 'bio-creep' with repeated NI trials, use of NI trials when determining whether excess safety risks can be ruled out, higher standards regarding quality of study conduct required with NI trials, and the myth that NI trials always require huge sample sizes.     

About half of the noninferiority trials tested superior treatments: a trial-register based study

ObjectivesA concern that noninferiority (NI) trials pose a risk of degradation of the treatment effects is prevalent. Thus, we aimed to determine the fraction of positive true effects (superiority rate) and the average true effect of current NI trials based on data from registered NI trials.Study Design and SettingAll NI trials carried out between 2000 and 2007 analyzing the NI of efficacy as the primary objective and registered in one of the two major clinical trials registers were studied. Having retrieved results from these trials, random effects modeling of the effect estimates was performed to determine the distribution of true effects.ResultsEffect estimates were available for 79 of 99 eligible trials identified. For trials with binary outcome, we estimated a superiority rate of 49% (95% confidence interval = 27–70%) and a mean true log odds ratio of ?0.005 (?0.112, 0.102). For trials with continuous outcome, the superiority rate was 58% (41–74%) and the mean true effect as Cohen's d of 0.06 (?0.064, 0.192).ConclusionsThe unanticipated finding of a positive average true effect and superiority of the new treatment in most NI trials suggest that the current practice of choosing NI designs in clinical trials makes degradation on average unlikely. However, the distribution of true treatment effects demonstrates that, in some NI trials, the new treatment is distinctly inferior.     

Reporting of noninferiority trials was incomplete in trial registries

Objective

To examine the registration of noninferiority trials, with a focus on the reporting of study design and noninferiority margins.

Study Design and Setting

Cross-sectional study of registry records of noninferiority trials published from 2005 to 2009 and records of noninferiority trials in the International Standard Randomized Controlled Trial Number (ISRCTN) or ClinicalTrials.gov trial registries. The main outcome was the proportion of records that reported the noninferiority design and margin.

Results

We analyzed 87 registry records of published noninferiority trials and 149 registry records describing noninferiority trials. Thirty-five (40%) of 87 records from published trials described the trial as a noninferiority trial; only two (2%) reported the noninferiority margin. Reporting of the noninferiority design was more frequent in the ISRCTN registry (13 of 18 records, 72%) compared with ClinicalTrials.gov (22 of 69 records, 32%; P = 0.002). Among the 149 records identified in the registries, 13 (9%) reported the noninferiority margin. Only one of the industry-sponsored trial compared with 11 of the publicly funded trials reported the margin (P = 0.001).

Conclusion

Most registry records of noninferiority trials do not mention the noninferiority design and do not include the noninferiority margin. The registration of noninferiority trials is unsatisfactory and must be improved.     

The impact of covariate adjustment at randomization and analysis for binary outcomes: understanding differences between superiority and noninferiority trials

The question of when to adjust for important prognostic covariates often arises in the design of clinical trials, and there remain various opinions on whether to adjust during both randomization and analysis, at randomization alone, or at analysis alone. Furthermore, little is known about the impact of covariate adjustment in the context of noninferiority (NI) designs. The current simulation‐based research explores this issue in the NI setting, as compared with the typical superiority setting, by assessing the differential impact on power, type I error, and bias in the treatment estimate as well as its standard error, in the context of logistic regression under both simple and covariate adjusted permuted block randomization algorithms. In both the superiority and NI settings, failure to adjust for covariates that influence outcome in the analysis phase, regardless of prior adjustment at randomization, results in treatment estimates that are biased toward zero, with standard errors that are deflated. However, as no treatment difference is approached under the null hypothesis in superiority and under the alternative in NI, this results in decreased power and nominal or conservative (deflated) type I error in the context of superiority but inflated power and type I error under NI. Results from the simulation study suggest that, regardless of the use of the covariate in randomization, it is appropriate to adjust for important prognostic covariates in analysis, as this yields nearly unbiased estimates of treatment as well as nominal type I error. Copyright © 2015 John Wiley & Sons, Ltd.     

Noninferiority hypotheses and choice of noninferiority margin

Ng (Drug Inf. J. 1993; 27:705-719; Drug Inf. J. 2001; 35:1517-1527) proposed that the noninferiority (NI) margin should be a small fraction of the therapeutic effect of the active control as compared with placebo in the setting of testing the NI hypothesis of the mean difference with a continuous outcome. For testing the NI hypothesis of the mean ratio with a continuous outcome, a similar NI margin on a log scale is proposed. This approach may also be applied in the setting of testing the NI hypotheses for survival data based on hazard ratios. Some pitfalls of testing the NI hypotheses with binary endpoints based on the difference or the ratio of proportions will be discussed. Testing the NI hypothesis with binary endpoints based on the odds ratio is proposed.     

基于相对度量指标的非劣效界值计算方法的比较

目的美国FDA在其非劣效临床试验指南中提供了两种基于相对度量指标的非劣效界值计算方法,一种是与绝对度量指标非劣效界值计算方法相对应的对数转换法(简称对数转换法),另外一种是直接基于相对风险的计算方法(简称直接计算法)。本文探讨了这两种方法计算结果的差异,以评估该差异对非劣效试验结果的影响。方法以风险比(hazard ratio,HR)为例,指代阳性对照药与安慰剂的疗效差异的保守估计值,从计算公式上阐述两种方法的关系,并展示两种方法在高优与低优指标中计算得到的非劣效界值的差异。结果当HR在0.80~1.25时,两种方法计算得到的非劣效界值差别在1%以内;当HR越远离1,差别越大。当取相同的效应保留比例f时,直接计算法计算出的非劣效界值总大于对数转换法,此时会导致低优指标使用直接计算法计算出的界值相对更激进,而高优指标则相对保守。当设定相同的非劣效界值δ时,对于低优指标,使用直接计算法所需设定的效应保留比例f高于对数转换法,对于高优指标则相反。结论在以相对度量指标作为主要评价指标的非劣效试验中,研究者应该认识到这两种计算方法对非劣效界值设定和试验结论的影响,综合考虑临床实践和试验药物的获益-风险等,慎重选取非劣效界值。     

Preliminary Safety and Efficacy of L‐carnitine Infusion for the Treatment of Vasopressor‐Dependent Septic Shock

Background: Sepsis is characterized by metabolic disturbances, and previous data suggest a relative carnitine deficiency may contribute to metabolic dysfunction. Studies regarding safety and patient‐centered efficacy of carnitine during septic shock are lacking. Methods: This was a double‐blind randomized control trial of levocarnitine (L‐carnitine) infusion vs normal saline for the treatment of vasopressor‐dependent septic shock. Patients meeting consensus definition for septic shock with a cumulative vasopressor index ≥3 and sequential organ failure assessment (SOFA) score ≥5 enrolled within 16 hours of the recognition of septic shock were eligible. The primary safety outcome was difference in serious adverse events (SAEs) per patient between groups. Efficacy outcomes included proportion of patients demonstrating a decrease in SOFA score of 2 or more points at 24 hours and short‐ and long‐term survival. Results: Of the 31 patients enrolled, 16 were in the L‐carnitine and 15 were in the placebo arm. There was no difference in SAEs between placebo and intervention (2.1 vs 1.8 SAEs per patient, P = .44). There was no difference in the proportion of patients achieving a decrease in SOFA score of 2 or more points at 24 hours between placebo and treatment (53% vs 44%, P = .59). Mortality was significantly lower at 28 days in the L‐carnitine group (4/16 vs 9/15, P = .048), with a nonsignificant improved survival at 1 year (P = .06). Conclusion: L‐carnitine infusion appears safe in vasopressor‐dependent septic shock. Preliminary efficacy data suggest potential benefit of L‐carnitine treatment, and further testing is indicated.     

Assessing the influence of treatment nonadherence on noninferiority trials using the tipping point approach

In noninferiority (NI) trials, an ongoing methodological challenge is how to handle in the analysis the subjects who are nonadherent to their assigned treatment. Some investigators perform the intent-to-treat (ITT) as the primary analysis and the per-protocol (PP) analysis as sensitivity analysis, whereas others do the reverse since ITT results may be anticonservative in the NI setting. But even when there is agreement between the ITT and PP approaches, NI of the experimental therapy to the comparator is not guaranteed. We propose that a tipping point method be used to further assess the impact of nonadherence on the results of a NI trial. In this approach, data from the nonadherers obtained after treatment discontinuation is not used, and their outcomes under the counterfactual situation of complete adherence are considered missing. The tipping point analysis indicates how sensitive the NI trial results are to the values of these missing counterfactual outcomes. The advantages of this approach are that a model or mechanism for the missing outcomes does not have to be assumed, and all subjects who were randomized are included in the analysis. We consider both binary and continuous outcomes and propose extensions to accommodate different types of nonadherence. The methods are illustrated with examples from two NI trials, one to evaluate different doses of radiation therapy to treat painful bone metastases and the other to compare treatments for reducing depression in adolescents.     

Designing multi‐arm multi‐stage clinical trials using a risk–benefit criterion for treatment selection

Multi‐arm clinical trials that compare several active treatments to a common control have been proposed as an efficient means of making an informed decision about which of several treatments should be evaluated further in a confirmatory study. Additional efficiency is gained by incorporating interim analyses and, in particular, seamless Phase II/III designs have been the focus of recent research. Common to much of this work is the constraint that selection and formal testing should be based on a single efficacy endpoint, despite the fact that in practice, safety considerations will often play a central role in determining selection decisions. Here, we develop a multi‐arm multi‐stage design for a trial with an efficacy and safety endpoint. The safety endpoint is explicitly considered in the formulation of the problem, selection of experimental arm and hypothesis testing. The design extends group‐sequential ideas and considers the scenario where a minimal safety requirement is to be fulfilled and the treatment yielding the best combined safety and efficacy trade‐off satisfying this constraint is selected for further testing. The treatment with the best trade‐off is selected at the first interim analysis, while the whole trial is allowed to compose of J analyses. We show that the design controls the familywise error rate in the strong sense and illustrate the method through an example and simulation. We find that the design is robust to misspecification of the correlation between the endpoints and requires similar numbers of subjects to a trial based on efficacy alone for moderately correlated endpoints. © 2015 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.     

Bayesian methods for setting sample sizes and choosing allocation ratios in phase II clinical trials with time‐to‐event endpoints

Conventional phase II trials using binary endpoints as early indicators of a time‐to‐event outcome are not always feasible. Uveal melanoma has no reliable intermediate marker of efficacy. In pancreatic cancer and viral clearance, the time to the event of interest is short, making an early indicator unnecessary. In the latter application, Weibull models have been used to analyse corresponding time‐to‐event data. Bayesian sample size calculations are presented for single‐arm and randomised phase II trials assuming proportional hazards models for time‐to‐event endpoints. Special consideration is given to the case where survival times follow the Weibull distribution. The proposed methods are demonstrated through an illustrative trial based on uveal melanoma patient data. A procedure for prior specification based on knowledge or predictions of survival patterns is described. This enables investigation into the choice of allocation ratio in the randomised setting to assess whether a control arm is indeed required. The Bayesian framework enables sample sizes consistent with those used in practice to be obtained. When a confirmatory phase III trial will follow if suitable evidence of efficacy is identified, Bayesian approaches are less controversial than for definitive trials. In the randomised setting, a compromise for obtaining feasible sample sizes is a loss in certainty in the specified hypotheses: the Bayesian counterpart of power. However, this approach may still be preferable to running a single‐arm trial where no data is collected on the control treatment. This dilemma is present in most phase II trials, where resources are not sufficient to conduct a definitive trial. Copyright © 2015 John Wiley & Sons, Ltd.     

Phase II clinical trials with time‐to‐event endpoints: optimal two‐stage designs with one‐sample log‐rank test

Phase II clinical trials are often conducted to determine whether a new treatment is sufficiently promising to warrant a major controlled clinical evaluation against a standard therapy. We consider single‐arm phase II clinical trials with right censored survival time responses where the ordinary one‐sample logrank test is commonly used for testing the treatment efficacy. For planning such clinical trials, this paper presents two‐stage designs that are optimal in the sense that the expected sample size is minimized if the new regimen has low efficacy subject to constraints of the type I and type II errors. Two‐stage designs, which minimize the maximal sample size, are also determined. Optimal and minimax designs for a range of design parameters are tabulated along with examples. Copyright © 2013 John Wiley & Sons, Ltd.     

A covariate‐adjustment regression model approach to noninferiority margin definition

To maintain the interpretability of the effect of experimental treatment (EXP) obtained from a noninferiority trial, current statistical approaches often require the constancy assumption. This assumption typically requires that the control treatment effect in the population of the active control trial is the same as its effect presented in the population of the historical trial. To prevent constancy assumption violation, clinical trial sponsors were recommended to make sure that the design of the active control trial is as close to the design of the historical trial as possible. However, these rigorous requirements are rarely fulfilled in practice. The inevitable discrepancies between the historical trial and the active control trial have led to debates on many controversial issues. Without support from a well‐developed quantitative method to determine the impact of the discrepancies on the constancy assumption violation, a correct judgment seems difficult. In this paper, we present a covariate‐adjustment generalized linear regression model approach to achieve two goals: (1) to quantify the impact of population difference between the historical trial and the active control trial on the degree of constancy assumption violation and (2) to redefine the active control treatment effect in the active control trial population if the quantification suggests an unacceptable violation. Through achieving goal (1), we examine whether or not a population difference leads to an unacceptable violation. Through achieving goal (2), we redefine the noninferiority margin if the violation is unacceptable. This approach allows us to correctly determine the effect of EXP in the noninferiority trial population when constancy assumption is violated due to the population difference. We illustrate the covariate‐adjustment approach through a case study. Copyright © 2010 John Wiley & Sons, Ltd.     

A causal model for longitudinal randomised trials with time‐dependent non‐compliance

In the presence of non‐compliance, conventional analysis by intention‐to‐treat provides an unbiased comparison of treatment policies but typically under‐estimates treatment efficacy. With all‐or‐nothing compliance, efficacy may be specified as the complier‐average causal effect (CACE), where compliers are those who receive intervention if and only if randomised to it. We extend the CACE approach to model longitudinal data with time‐dependent non‐compliance, focusing on the situation in which those randomised to control may receive treatment and allowing treatment effects to vary arbitrarily over time. Defining compliance type to be the time of surgical intervention if randomised to control, so that compliers are patients who would not have received treatment at all if they had been randomised to control, we construct a causal model for the multivariate outcome conditional on compliance type and randomised arm. This model is applied to the trial of alternative regimens for glue ear treatment evaluating surgical interventions in childhood ear disease, where outcomes are measured over five time points, and receipt of surgical intervention in the control arm may occur at any time. We fit the models using Markov chain Monte Carlo methods to obtain estimates of the CACE at successive times after receiving the intervention. In this trial, over a half of those randomised to control eventually receive intervention. We find that surgery is more beneficial than control at 6months, with a small but non‐significant beneficial effect at 12months. © 2015 The Authors. Statistics in Medicine Published by JohnWiley & Sons Ltd.     

Simultaneous sequential monitoring of efficacy and safety led to masking of effects

ObjectiveUsually, sequential designs for clinical trials are applied on the primary (=efficacy) outcome. In practice, other outcomes (e.g., safety) will also be monitored and influence the decision whether to stop a trial early. Implications of simultaneous monitoring on trial decision making are yet unclear. This study examines what happens to the type I error, power, and required sample sizes when one efficacy outcome and one correlated safety outcome are monitored simultaneously using sequential designs.Study Design and SettingWe conducted a simulation study in the framework of a two-arm parallel clinical trial. Interim analyses on two outcomes were performed independently and simultaneously on the same data sets using four sequential monitoring designs, including O'Brien-Fleming and Triangular Test boundaries. Simulations differed in values for correlations and true effect sizes.ResultsWhen an effect was present in both outcomes, competition was introduced, which decreased power (e.g., from 80% to 60%). Futility boundaries for the efficacy outcome reduced overall type I errors as well as power for the safety outcome.ConclusionMonitoring two correlated outcomes, given that both are essential for early trial termination, leads to masking of true effects. Careful consideration of scenarios must be taken into account when designing sequential trials. Simulation results can help guide trial design.     

Blinded sample size re‐estimation in three‐arm trials with ‘gold standard’ design

In this article, we study blinded sample size re‐estimation in the ‘gold standard’ design with internal pilot study for normally distributed outcomes. The ‘gold standard’ design is a three‐arm clinical trial design that includes an active and a placebo control in addition to an experimental treatment. We focus on the absolute margin approach to hypothesis testing in three‐arm trials at which the non‐inferiority of the experimental treatment and the assay sensitivity are assessed by pairwise comparisons. We compare several blinded sample size re‐estimation procedures in a simulation study assessing operating characteristics including power and type I error. We find that sample size re‐estimation based on the popular one‐sample variance estimator results in overpowered trials. Moreover, sample size re‐estimation based on unbiased variance estimators such as the Xing–Ganju variance estimator results in underpowered trials, as it is expected because an overestimation of the variance and thus the sample size is in general required for the re‐estimation procedure to eventually meet the target power. To overcome this problem, we propose an inflation factor for the sample size re‐estimation with the Xing–Ganju variance estimator and show that this approach results in adequately powered trials. Because of favorable features of the Xing–Ganju variance estimator such as unbiasedness and a distribution independent of the group means, the inflation factor does not depend on the nuisance parameter and, therefore, can be calculated prior to a trial. Moreover, we prove that the sample size re‐estimation based on the Xing–Ganju variance estimator does not bias the effect estimate. Copyright © 2017 John Wiley & Sons, Ltd.     

Hierarchical nested trial design (HNTD) for demonstrating treatment efficacy of new antibacterial drugs in patient populations with emerging bacterial resistance

In the last decade or so, pharmaceutical drug development activities in the area of new antibacterial drugs for treating serious bacterial diseases have declined, and at the same time, there are worries that the increased prevalence of antibiotic‐resistant bacterial infections, especially the increase in drug‐resistant Gram‐negative infections, limits available treatment options . A recent CDC report, ‘Antibiotic Resistance Threats in the United States’, indicates that antimicrobial resistance is one of our most serious health threats. However, recently, new ideas have been proposed to change this situation. An idea proposed in this regard is to conduct randomized clinical trials in which some patients, on the basis of a diagnostic test, may show presence of bacterial pathogens that are resistant to the control treatment, whereas remaining patients would show pathogens that are susceptible to the control. The control treatment in such trials can be the standard of care or the best available therapy approved for the disease. Patients in the control arm with resistant pathogens can have the option for rescue therapies if their clinical signs and symptoms worsen. A statistical proposal for such patient populations is to use a hierarchical noninferiority–superiority nested trial design that is informative and allows for treatment‐to‐control comparisons for the two subpopulations without any statistical penalty. This design can achieve in the same trial dual objectives: (i) to show that the new drug is effective for patients with susceptible pathogens on the basis of a noninferiority test and (ii) to show that it is superior to the control in patients with resistant pathogens. This paper addresses statistical considerations and methods for achieving these two objectives for this design. Published 2014. This article is a U.S. Government work and is in the public domain in the USA.