Design and analysis of non-inferiority mortality trials in oncology

The recent revision of the Declaration of Helsinki and the existence of many new therapies that affect survival or serious morbidity, and that therefore cannot be denied patients, have generated increased interest in active-control trials, particularly those intended to show equivalence or non-inferiority to the active-control. A non-inferiority hypothesis has historically been formulated in terms of a fixed margin. This margin was historically designed to exclude a 'clinically meaningful difference', but has become recognized that the margin must also be no larger than the assured effect of the control in the new study. Depending on how this 'assured effect' is determined or estimated, the selected margin may be very small, leading to very large sample sizes, especially when there is an added requirement that a loss of some specified fraction of the assured effect must be ruled out. In cases where it is appropriate, this paper proposes non-inferiority analyses that do not involve a fixed margin, but can be described as a two confidence interval procedure that compares the 95 per cent two-sided CI for the difference between the treatment and the control to a confidence interval for the control effect (based on a meta-analysis of historical data comparing the control to placebo) that is chosen to preserve a study-wide type I error rate of about 0.025 (similar to the usual standard for a superiority trial) for testing for retention of a prespecified fraction of the control effect. The approach assumes that the estimate of the historical active-control effect size is applicable in the current study. If there is reason to believe that this effect size is diminished (for example, improved concomitant therapies) the estimate of this historical effect could be reduced appropriately. The statistical methodology for testing this non-inferiority hypothesis is developed for a hazard ratio (rather than an absolute difference between treatments, because a hazard ratio seems likely to be less population dependent than the absolute difference). In the case of oncology, the hazard ratio is the usual way of comparing treatments with respect to time to event (time to progression or survival) endpoints. The proportional hazards assumption is regarded as reasonable (approximately holding). The testing procedures proposed are conditionally equivalent to two confidence interval procedures that relax the conservatism of two 95 per cent confidence interval testing procedures and preserve the type I error rate at a one-sided 0.025 level. An application of this methodology to Xeloda, a recently approved drug for the treatment of metastatic colorectal cancers, is illustrated. Other methodologies are also described and assessed - including a point estimate procedure, a Bayesian procedure and two delta-method confidence interval procedures. Published in 2003 by John Wiley & Sons, Ltd.     

A note on the conservativeness of the confidence interval approach for the selection of non-inferiority margin in the two-arm active-control trial

Compared with placebo-control clinical trials, the interpretation of efficacy results from active-control trials requires more caution. This is because efficacy results from such trials cannot be reliably interpreted without a thorough understanding of the efficacy evidence that formed the basis for the approval of the active control, especially when such drug efficacy is to be established on the basis of clinical evidence from the traditional two-arm active-control clinical equivalence studies as opposed to the multi-arm active control. This is because in addition to over-reliance on the quantification of a clinically irrelevant acceptable margin of inferiority from historical data, such interpretation also depends on cross-trial inference for demonstration of experimental drug effect. We provide a brief overview of some design issues with the traditional two-arm active-control clinical trial and discuss regulators' concern regarding Type I error rate control (with the two most popular methods for the quantification of the non-inferiority margin) in cross-trial demonstration of experimental drug effect. Simulation results are presented to show that the point estimate method provides adequate control of the Type I error rate with > or =75 per cent retention of known active-control effect and that the confidence interval approach is uniformly ultra-conservative. We also report (via a numerical example from real clinical trial data) a couple of potentially less stringent alternative approaches for establishing the non-inferiority of a test drug over a control, which have been used in the past to provide additional efficacy evidence in NDA submission.     

TACT method for non-inferiority testing in active controlled trials

In active controlled trials without a placebo arm, non-inferiority testing is often considered but has different objectives. For the objective of demonstrating the efficacy of an experimental treatment or retention of a fraction of the control effect by the treatment, there are two types of statistical methods for testing - the synthesis method and the confidence interval method. According to the study of Wang, Hung and Tsong, the former is efficient under the so-called constancy condition but may have the alpha error rate inflate rapidly if the condition does not hold. In contrast, the latter method with careful selection of the non-inferiority margin tends to be conservative if the condition holds and may still have a valid alpha error otherwise unless the effect of the active control is less to a large extent in the active controlled trial than in the historical trials. We developed the TACT method, Two-stage Active Control Testing, as a viable compromise between the two methods. Through the TACT method, the uninterpretable non-inferiority testing may be avoided prior to the end of the trial. The TACT method carefully constructed can have a valid alpha error rate and the power close to the synthesis method if the constancy condition holds. In addition, the TACT method is more powerful than the confidence interval method for testing for the efficacy of the new treatment relative to the putative placebo and for showing that the new treatment is not inferior to the active control comparator.     

A proof of concept phase II non-inferiority criterion

Traditional phase III non-inferiority trials require compelling evidence that the treatment vs control effect bfθ is better than a pre-specified non-inferiority margin θ(NI) . The standard approach compares this margin to the 95 per cent confidence interval of the effect parameter. In the phase II setting, in order to declare Proof of Concept (PoC) for non-inferiority and proceed in the development of the drug, different criteria that are specifically tailored toward company internal decision making may be more appropriate. For example, less evidence may be needed as long as the effect estimate is reasonably convincing. We propose a non-inferiority design that addresses the specifics of the phase II setting. The requirements are that (1) the effect estimate be better than a critical threshold θ(C), and (2) the type I error with regard to θ(NI) is controlled at a pre-specified level. This design is compared with the traditional design from a frequentist as well as a Bayesian perspective, where the latter relies on the Level of Proof (LoP) metric, i.e. the probability that the true effect is better than effect values of interest. Clinical input is required to establish the value θ(C), which makes the design transparent and improves interactions within clinical teams. The proposed design is illustrated for a non-inferiority trial for a time-to-event endpoint in oncology.     

Controlling the type 1 error rate in non-inferiority trials

Two different approaches have been proposed for establishing the efficacy of an experimental therapy through a non-inferiority trial: The fixed-margin approach involves first defining a non-inferiority margin and then demonstrating that the experimental therapy is not worse than the control by more than this amount, and the synthesis approach involves combining the data from the non-inferiority trial with the data from historical trials evaluating the effect of the control. In this paper, we introduce a unified approach that has both these approaches as special cases and show how the parameters of this approach can be selected to control the unconditional type 1 error rate in the presence of departures from the assumptions of assay sensitivity and constancy. It is shown that the fixed-margin approach can be extremely inefficient and that it is always possible to achieve equivalent control of the unconditional type 1 error rate, with higher power, by using an appropriately chosen synthesis method.     

Modified Haybittle-Peto group sequential designs for testing superiority and non-inferiority hypotheses in clinical trials

In designing an active controlled clinical trial, one sometimes has to choose between a superiority objective (to demonstrate that a new treatment is more effective than an active control therapy) and a non-inferiority objective (to demonstrate that it is no worse than the active control within some pre-specified non-inferiority margin). It is often difficult to decide which study objective should be undertaken at the planning stage when one does not have actual data on the comparative advantage of the new treatment. By making use of recent advances in the theory of efficient group sequential tests, we show how this difficulty can be resolved by a flexible group sequential design that can adaptively choose between the superiority and non-inferiority objectives during interim analyses. While maintaining the type I error probability at a pre-specified level, the proposed test is shown to have power advantage and/or sample size saving over fixed sample size tests for either only superiority or non-inferiority, and over other group sequential designs in the literature.     

A two-stage sample size recalculation procedure for placebo- and active-controlled non-inferiority trials

Many non-inferiority trials of a test treatment versus an active control may also, if ethical, incorporate a placebo arm. Inclusion of a placebo arm enables a direct assessment of assay sensitivity. It also allows construction of a non-inferiority test that avoids the problematic specification of an absolute non-inferiority margin, and instead evaluates whether the test treatment preserves a pre-specified proportion of the effect of the active control over placebo. We describe a two-stage procedure for sample size recalculation in such a setting that maintains the desired power more closely than a fixed sample approach when the magnitude of the effect of the active control differs from that anticipated. We derive an allocation rule for randomization under which the procedure preserves the type I error rate, and show that this coincides with that previously presented for optimal allocation of the sample size among the three treatment arms.     

Group sequential test strategies for superiority and non-inferiority hypotheses in active controlled clinical trials.

In a group sequential active controlled clinical trial, the study hypothesis may be a superiority hypothesis that an experimental treatment is more effective than the active control therapy or a non-inferiority hypothesis that the treatment is no worse than the active control within some non-inferiority range. When it is necessary to plan for testing the superiority and the non-inferiority hypotheses, we propose an adaptive group sequential closed test strategy by which the sample size is planned for testing superiority and is to be increased for showing non-inferiority given that it is deemed more plausible than superiority based on the observed sample path during the course of the trial. The proposed adaptive test strategy is valid in terms of having the type I error probability maintained at the targeted alpha level for both superiority and non-inferiority. It has power advantage or sample size saving over the traditional group sequential test designed for testing either superiority only or non-inferiority only.     

Exact unconditional tests for testing non-inferiority in matched-pairs design

The problem of testing non-inferiority in a 2 x 2 matched-pairs sample is considered. Two exact unconditional tests based on the standard and the confidence interval p-values are proposed. Although tests of non-inferiority have two nuisance parameters under the null hypothesis, the exact tests are defined by reducing the dimension of nuisance parameter space from two to one using the monotonicity of the distribution. The exact sizes and powers of these tests and the existing asymptotic test are considered. The exact tests are found to be accurate in view of their size property. In addition, the exact test based on the confidence interval p-value is more powerful than the other exact test. It is shown that the asymptotic test is inaccurate, that is, its size exceeds the claimed nominal level alpha. Therefore, it recommends a cautious approach in use of the asymptotic test for the problem of testing non-inferiority, particularly when sample sizes are small or moderately large.     

Two-stage adaptive strategy for superiority and non-inferiority hypotheses in active controlled clinical trials

In active controlled trials without a placebo arm, there are usually two study objectives: to test a superiority hypothesis that the experimental treatment is more effective than the active control therapy, and to test a non-inferiority hypothesis that the experimental treatment is therapeutically no worse than the active control within a defined margin. For a two-stage adaptive design, it is not necessary to give a fixed sample size calculation at the planning stage of the study when treatment effect information is often insufficient. Instead, decision and estimation of the design specifications can be made more reliably after the first stage when interim results are available. We propose the use of conditional power approach to determine the sample size and critical values for testing the superiority and non-inferiority hypotheses for the second stage based on the observed result of the first stage. The proposed adaptive procedure preserves the overall type I error rate for both superiority and non-inferiority, and has the flexibility of early termination of the study (for futility or efficacy) or extending the study by appropriate sample size.     

On non-inferiority margin and statistical tests in active control trials

The problem of selecting a non-inferiority margin and the corresponding statistical test for non-inferiority in active control trials is considered. For selection of non-inferiority margin, the guideline by the International Conference on Harmonization (ICH) recommends that the non-inferiority margin should be chosen in such a way that if the non-inferiority of the test therapy to the active control agent is claimed, the test therapy is not only non-inferior to the active control agent, but also superior to the placebo. Furthermore, variability should be taken into account. Along this line, a method for selecting non-inferiority margins with some statistical justification is proposed. Statistical tests for non-inferiority designed in the situation where the non-inferiority margin is an unknown parameter are derived. An example concerning a cancer trail for testing non-inferiority with the primary study endpoint of the time to disease progression is presented to illustrate the proposed method.     

Statistical considerations of FDA and CPMP rules for the investigation of new anti-bacterial products

In this paper I investigate statistical properties of some guidance given by the FDA and by the CPMP on the planning, conduct and analysis of clinical trials with new anti-bacterial substances using an active control design. It is demonstrated that the non-inferiority margin proposed by the FDA has some undesirable features, and that the CPMP guidance may need further interpretation with respect to a statement that the non-inferiority margin may be smaller than 10 per cent for response rates >90 per cent. A new margin is proposed that combines the desirable properties from both the FDA and the CPMP guidance. It is also shown that the approximate unconditional tests that are in use in such trials are quite unreliable with respect to preserving the nominal type I error. Unconditional exact tests are presented as a remedy for this issue.     

Planning and analysis of three-arm non-inferiority trials with binary endpoints

Three-arm trials including an experimental treatment, an active control and a placebo group are frequently preferred for the assessment of non-inferiority. In contrast to two-arm non-inferiority studies, these designs allow a direct proof of efficacy of a new treatment by comparison with placebo. As a further advantage, the test problem for establishing non-inferiority can be formulated in such a way that rejection of the null hypothesis assures that a pre-defined portion of the (unknown) effect the reference shows versus placebo is preserved by the treatment under investigation. We present statistical methods for this study design and the situation of a binary outcome variable. Asymptotic test procedures are given and their actual type I error rates are calculated. Approximate sample size formulae are derived and their accuracy is discussed. Furthermore, the question of optimal allocation of the total sample size is considered. Power properties of the testing strategy including a pre-test for assay sensitivity are presented. The derived methods are illustrated by application to a clinical trial in depression.     

Strength of evidence of non‐inferiority trials—The adjustment of the type I error rate in non‐inferiority trials with the synthesis method

In non‐inferiority trials that employ the synthesis method several types of dependencies among test statistics occur due to sharing of the same information from the historical trial. The conditions under which the dependencies appear may be divided into three categories. The first case is when a new drug is approved with single non‐inferiority trial. The second case is when a new drug is approved if two independent non‐inferiority trials show positive results. The third case is when two new different drugs are approved with the same active control. The problem of the dependencies is that they can make the type I error rate deviate from the nominal level. In order to study such deviations, we introduce the unconditional and conditional across‐trial type I error rates when the non‐inferiority margin is estimated from the historical trial, and investigate how the dependencies affect the type I error rates. We show that the unconditional across‐trial type I error rate increases dramatically as does the correlation between two non‐inferiority tests when a new drug is approved based on the positive results of two non‐inferiority trials. We conclude that the conditional across‐trial type I error rate involves the unknown treatment effect in the historical trial. The formulae of the conditional across‐trial type I error rates provide us with a way of investigating the conditional across‐trial type I error rates for various assumed values of the treatment effect in the historical trial. Copyright © 2010 John Wiley & Sons, Ltd.     

Tests for equivalence or non-inferiority for paired binary data.

Assessment of therapeutic equivalence or non-inferiority between two medical diagnostic procedures often involves comparisons of the response rates between paired binary endpoints. The commonly used and accepted approach to assessing equivalence is by comparing the asymptotic confidence interval on the difference of two response rates with some clinical meaningful equivalence limits. This paper investigates two asymptotic test statistics, a Wald-type (sample-based) test statistic and a restricted maximum likelihood estimation (RMLE-based) test statistic, to assess equivalence or non-inferiority based on paired binary endpoints. The sample size and power functions of the two tests are derived. The actual type I error and power of the two tests are computed by enumerating the exact probabilities in the rejection region. The results show that the RMLE-based test controls type I error better than the sample-based test. To establish an equivalence between two treatments with a symmetric equivalence limit of 0.15, a minimal sample size of 120 is needed. The RMLE-based test without the continuity correction performs well at the boundary point 0. A numerical example illustrates the proposed procedures.     

On the three-arm non-inferiority trial including a placebo with a prespecified margin

Three-arm trials including the experimental treatment, an active reference treatment and a placebo are recommended in the guidelines of the ICH and EMEA/CPMP as a useful approach to the assessment of assay sensitivity. Generally, the acceptable non-inferiority margin Δ has been defined as the maximum clinically irrelevant difference between treatments in many two-arm non-inferiority trials. However, many recent articles discussing three-arm trials have considered a design with unknown Δ which is the prespecified fraction f of unknown effect size of the reference drug, where the prespecified fraction f is treated as if it were a revised margin. Therefore, these methods cannot be applied to the case where the acceptable non-inferiority margin must be a prespecified difference between treatments. In this paper, we propose a statistical test procedure for three-arm non-inferiority trials with the margin Δ defined as a prespecified difference between treatments under the situation that the primary endpoints are normally distributed with a common, but unknown, variance. In addition, we derive the optimal allocation that minimizes the total sample size. The proposed method is illustrated with data on a randomized controlled trial on major depressive disorder.     

Non-inferiority trials: design concepts and issues - the encounters of academic consultants in statistics

Placebo-controlled trials are the ideal for evaluating medical treatment efficacy. They allow for control of the placebo effect and are most efficient, requiring the smallest numbers of patients to detect a treatment effect. A placebo control is ethically justified if no standard treatment exists, if the standard treatment has not been proven efficacious, there are no risks associated with delaying treatment or escape clauses are included in the protocol. Where possible and justified, they should be the first choice for medical treatment evaluation. Given the large number of proven effective treatments, placebo-controlled trials are often unethical. In these situations active-controlled trials are generally appropriate. The non-inferiority trial is appropriate for evaluation of the efficacy of an experimental treatment versus an active control when it is hypothesized that the experimental treatment may not be superior to a proven effective treatment, but is clinically and statistically not inferior in effectiveness. These trials are not easy to design. An active control must be selected. Good historical placebo-controlled trials documenting the efficacy of the active control must exist. From these historical trials statistical analysis must be performed and clinical judgement applied in order to determine the non-inferiority margin M and to assess assay sensitivity. The latter refers to establishing that the active drug would be superior to the placebo in the setting of the present non-inferiority trial (that is, the constancy assumption). Further, a putative placebo analysis of the new treatment versus the placebo using data from the non-inferiority trial and the historical active versus placebo-controlled trials is needed. Useable placebo-controlled historical trials for the active control are often not available, and determination of assay sensitivity and an appropriate M is difficult and debatable. Serious consideration to expansions of and alternatives to non-inferiority trials are needed.     

Efficient adaptive designs with mid-course sample size adjustment in clinical trials

Adaptive designs have been proposed for clinical trials in which the nuisance parameters or alternative of interest are unknown or likely to be misspecified before the trial. Although most previous works on adaptive designs and mid-course sample size re-estimation have focused on two-stage or group-sequential designs in the normal case, we consider here a new approach that involves at most three stages and is developed in the general framework of multiparameter exponential families. This approach not only maintains the prescribed type I error probability but also provides a simple but asymptotically efficient sequential test whose finite-sample performance, measured in terms of the expected sample size and power functions, is shown to be comparable to the optimal sequential design, determined by dynamic programming, in the simplified normal mean case with known variance and prespecified alternative, and superior to the existing two-stage designs and also to adaptive group-sequential designs when the alternative or nuisance parameters are unknown or misspecified.     

Use of the equivalence approach in reproductive health clinical trials.

An equivalence trial is appropriate when it is desired to demonstrate equivalence between two treatments, regimens or interventions (methods) or non-inferiority of a new one compared to a standard one. The conduct of an equivalence trial requires different techniques during design and analysis compared to a superiority trial. The existing formulae for sample size calculation to demonstrate equivalence between two methods using the confidence interval approach are reviewed. The establishment of the margin of equivalence and the choice of the type of test are discussed. Plots of sample sizes required to demonstrate equivalence in the case of binary outcomes are presented for values of proportions and margins of equivalence common in the reproductive health field. Examples are given of method comparisons in the reproductive health field in which the relevant question is to demonstrate non-inferiority. The approach to equivalence is described in the trials included in three published systematic reviews in which these comparisons were conducted, addressing the statement of hypotheses, sample size calculation and the interpretation of results. The use of the conventional superiority approach to design equivalence trials has led to underpowered trials to show equivalence within clinical relevant margins. The analysis and interpretation of results from such trials has resulted in conclusions of equivalence based on lack of significance. We draw attention to the lack of awareness of the appropriate techniques for equivalence trials among researchers in the field of reproductive health. Finally, the issue of interim analyses and stopping rules in equivalence trials is addressed.     

The use of putative placebo in active control trials: two applications in a regulatory setting

For life-threatening diseases, ethical considerations preclude the inclusion of an untreated control group in the investigation of a new therapeutic agent when a standard therapy exists. In these cases, active controlled studies are conducted, and may be planned to demonstrate either superiority or equivalence/non-inferiority of the new drug over the standard therapy (active control). In the non-inferiority study, an important aspect is the ability to detect an inferior drug (assay sensitivity). It has been suggested that assay sensitivity for a non-inferiority study should be deduced from historical data, specifically placebo controlled studies with the standard therapy. The assessment of assay sensitivity may also be important in a superiority trial that fails to demonstrate a statistically significant difference between treatments, and the sponsor attempts to determine whether there is lack of inferiority as an alternative hypothesis for regulatory approval. This paper describes two methods of putative placebo analysis for assessing assay sensitivity in active controlled trials. One approach imputes a point estimate for the odds ratio (95 per cent confidence interval) for a new drug (T) compared to a placebo control (P). A Bayesian approach calculates the posterior probability that T is superior to P, or, that T is at least k per cent as good as the active control (A) and A is more effective than P. These methods are applied in two clinical/regulatory settings: a phase III trial comparing docetaxel (Taxotere) to doxorubicin in metastatic breast cancer patients, and a phase III programme with two trials comparing enoxaparin (Lovenox) plus aspirin to unfractionated heparin plus aspirin in patients with unstable angina or non-Q-wave myocardial infarction. The methodologies presented in this paper were used in securing regulatory approval for docetaxel in the treatment of locally advanced or metastatic breast cancer after failure of prior chemotherapy, and for enoxaparin in the treatment of acute coronary syndrome.