A note on the conservativeness of the confidence interval approach for the selection of non-inferiority margin in the two-arm active-control trial

Compared with placebo-control clinical trials, the interpretation of efficacy results from active-control trials requires more caution. This is because efficacy results from such trials cannot be reliably interpreted without a thorough understanding of the efficacy evidence that formed the basis for the approval of the active control, especially when such drug efficacy is to be established on the basis of clinical evidence from the traditional two-arm active-control clinical equivalence studies as opposed to the multi-arm active control. This is because in addition to over-reliance on the quantification of a clinically irrelevant acceptable margin of inferiority from historical data, such interpretation also depends on cross-trial inference for demonstration of experimental drug effect. We provide a brief overview of some design issues with the traditional two-arm active-control clinical trial and discuss regulators' concern regarding Type I error rate control (with the two most popular methods for the quantification of the non-inferiority margin) in cross-trial demonstration of experimental drug effect. Simulation results are presented to show that the point estimate method provides adequate control of the Type I error rate with > or =75 per cent retention of known active-control effect and that the confidence interval approach is uniformly ultra-conservative. We also report (via a numerical example from real clinical trial data) a couple of potentially less stringent alternative approaches for establishing the non-inferiority of a test drug over a control, which have been used in the past to provide additional efficacy evidence in NDA submission.     

On the three-arm non-inferiority trial including a placebo with a prespecified margin

Three-arm trials including the experimental treatment, an active reference treatment and a placebo are recommended in the guidelines of the ICH and EMEA/CPMP as a useful approach to the assessment of assay sensitivity. Generally, the acceptable non-inferiority margin Δ has been defined as the maximum clinically irrelevant difference between treatments in many two-arm non-inferiority trials. However, many recent articles discussing three-arm trials have considered a design with unknown Δ which is the prespecified fraction f of unknown effect size of the reference drug, where the prespecified fraction f is treated as if it were a revised margin. Therefore, these methods cannot be applied to the case where the acceptable non-inferiority margin must be a prespecified difference between treatments. In this paper, we propose a statistical test procedure for three-arm non-inferiority trials with the margin Δ defined as a prespecified difference between treatments under the situation that the primary endpoints are normally distributed with a common, but unknown, variance. In addition, we derive the optimal allocation that minimizes the total sample size. The proposed method is illustrated with data on a randomized controlled trial on major depressive disorder.     

A two-stage sample size recalculation procedure for placebo- and active-controlled non-inferiority trials

Many non-inferiority trials of a test treatment versus an active control may also, if ethical, incorporate a placebo arm. Inclusion of a placebo arm enables a direct assessment of assay sensitivity. It also allows construction of a non-inferiority test that avoids the problematic specification of an absolute non-inferiority margin, and instead evaluates whether the test treatment preserves a pre-specified proportion of the effect of the active control over placebo. We describe a two-stage procedure for sample size recalculation in such a setting that maintains the desired power more closely than a fixed sample approach when the magnitude of the effect of the active control differs from that anticipated. We derive an allocation rule for randomization under which the procedure preserves the type I error rate, and show that this coincides with that previously presented for optimal allocation of the sample size among the three treatment arms.     

Two-stage adaptive strategy for superiority and non-inferiority hypotheses in active controlled clinical trials

In active controlled trials without a placebo arm, there are usually two study objectives: to test a superiority hypothesis that the experimental treatment is more effective than the active control therapy, and to test a non-inferiority hypothesis that the experimental treatment is therapeutically no worse than the active control within a defined margin. For a two-stage adaptive design, it is not necessary to give a fixed sample size calculation at the planning stage of the study when treatment effect information is often insufficient. Instead, decision and estimation of the design specifications can be made more reliably after the first stage when interim results are available. We propose the use of conditional power approach to determine the sample size and critical values for testing the superiority and non-inferiority hypotheses for the second stage based on the observed result of the first stage. The proposed adaptive procedure preserves the overall type I error rate for both superiority and non-inferiority, and has the flexibility of early termination of the study (for futility or efficacy) or extending the study by appropriate sample size.     

A hybrid Bayesian-frequentist approach to evaluate clinical trial designs for tests of superiority and non-inferiority

Specification of the study objective of superiority or non-inferiority at the design stage of a phase III clinical trial can sometimes be very difficult due to the uncertainty that surrounds the efficacy level of the experimental treatment. This uncertainty makes it tempting for investigators to design a trial that would allow testing of both superiority and non-inferiority hypotheses. However, when a conventional single-stage design is used to test both hypotheses, the sample size is based on the chosen primary objective of either superiority or non-inferiority. In this situation, the power of the test for the secondary objective can be low, which may lead to a large loss of resources. Potentially low reproducibility is another major concern for the single-stage design in phase III trials, because significant findings of confirmatory trials are required to be reproducible. In this paper, we propose a hybrid Bayesian-frequentist approach to evaluate reproducibility and power in single-stage designs for phase III trials to test both superiority and non-inferiority. The essence of the proposed approach is to express the uncertainty that surrounds the efficacy of the experimental treatment as a probability distribution. Then one can use Bayes formula with simple graphical techniques to evaluate reproducibility and power adequacy.     

Assessing non-inferiority: a combination approach

Non-inferiority designs are growing in importance as a strategy for comparing new drugs with established therapies. Because it is not possible to show that a new drug and the established therapy have identical efficacy profiles, non-inferiority trials are designed to demonstrate that the new drug is not inferior to an established drug (the 'control') relative to a prespecified 'non-inferiority margin'. No objective principle guides the choice of the non-inferiority margin, and controversies about the margin have, in some cases, had important consequences for drug development.We argue that some of these controversies have arisen because non-inferiority trials must achieve two objectives. They must demonstrate not only that the new drug is not inferior to the control drug by the non-inferiority margin, but also that the new drug is superior to placebo. When the second objective is not considered explicitly, it can distort the choice of the non-inferiority margin. Some methods designed to address both objectives through the choice of the non-inferiority margin lead to overly stringent non-inferiority criteria.We describe an approach to non-inferiority analysis that combines two tests, a traditional test for non-inferiority and a test for superiority based on a synthetic estimate of the effect of the new treatment relative to placebo. The synthetic estimate may be 'discounted' to address concerns about assay inconstancy. We discuss power and sample size considerations for the proposed procedure.     

Planning and analysis of three-arm non-inferiority trials with binary endpoints

Three-arm trials including an experimental treatment, an active control and a placebo group are frequently preferred for the assessment of non-inferiority. In contrast to two-arm non-inferiority studies, these designs allow a direct proof of efficacy of a new treatment by comparison with placebo. As a further advantage, the test problem for establishing non-inferiority can be formulated in such a way that rejection of the null hypothesis assures that a pre-defined portion of the (unknown) effect the reference shows versus placebo is preserved by the treatment under investigation. We present statistical methods for this study design and the situation of a binary outcome variable. Asymptotic test procedures are given and their actual type I error rates are calculated. Approximate sample size formulae are derived and their accuracy is discussed. Furthermore, the question of optimal allocation of the total sample size is considered. Power properties of the testing strategy including a pre-test for assay sensitivity are presented. The derived methods are illustrated by application to a clinical trial in depression.     

Power and sample size computations in simultaneous tests for non-inferiority based on relative margins

In this paper, we address the problem of calculating power and sample sizes associated with simultaneous tests for non-inferiority. We consider the case of comparing several experimental treatments with an active control. The approach is based on the ratio view, where the common non-inferiority margin is chosen to be some percentage of the mean of the control treatment. Two power definitions in multiple hypothesis testing, namely, complete power and minimal power, are used in the computations. The sample sizes associated with the ratio-based inference are also compared with that of a comparable inference based on the difference of means for various scenarios. It is found that the sample size required for ratio-based inferences is smaller than that of difference-based inferences when the relative non-inferiority margin is less than one and when large response values indicate better treatment effects. The results are illustrated with examples.     

On non-inferiority margin and statistical tests in active control trials

The problem of selecting a non-inferiority margin and the corresponding statistical test for non-inferiority in active control trials is considered. For selection of non-inferiority margin, the guideline by the International Conference on Harmonization (ICH) recommends that the non-inferiority margin should be chosen in such a way that if the non-inferiority of the test therapy to the active control agent is claimed, the test therapy is not only non-inferior to the active control agent, but also superior to the placebo. Furthermore, variability should be taken into account. Along this line, a method for selecting non-inferiority margins with some statistical justification is proposed. Statistical tests for non-inferiority designed in the situation where the non-inferiority margin is an unknown parameter are derived. An example concerning a cancer trail for testing non-inferiority with the primary study endpoint of the time to disease progression is presented to illustrate the proposed method.     

A combined superiority and non-inferiority approach to multiple endpoints in clinical trials

Treatment comparisons in clinical trials often involve multiple endpoints. By making use of bootstrap tests, we develop a new non-parametric approach to multiple-endpoint testing that can be used to demonstrate non-inferiority of a new treatment for all endpoints and superiority for some endpoint when it is compared to an active control. It is shown that this approach does not incur a large multiplicity cost in sample size to achieve reasonable power and that it can incorporate complex dependencies in the multivariate distributions of all outcome variables for the two treatments via bootstrap resampling.     

Modified Haybittle-Peto group sequential designs for testing superiority and non-inferiority hypotheses in clinical trials

In designing an active controlled clinical trial, one sometimes has to choose between a superiority objective (to demonstrate that a new treatment is more effective than an active control therapy) and a non-inferiority objective (to demonstrate that it is no worse than the active control within some pre-specified non-inferiority margin). It is often difficult to decide which study objective should be undertaken at the planning stage when one does not have actual data on the comparative advantage of the new treatment. By making use of recent advances in the theory of efficient group sequential tests, we show how this difficulty can be resolved by a flexible group sequential design that can adaptively choose between the superiority and non-inferiority objectives during interim analyses. While maintaining the type I error probability at a pre-specified level, the proposed test is shown to have power advantage and/or sample size saving over fixed sample size tests for either only superiority or non-inferiority, and over other group sequential designs in the literature.     

Non-inferiority trials: the 'at least as good as' criterion with dichotomous data

The 'at least as good as' criterion, introduced by Laster and Johnson for a continuous response variate, is developed here for applications with dichotomous data. This approach is adaptive in nature, as the margin of non-inferiority is not taken as a fixed difference; it varies as a function of the positive control response. When the non-inferiority margin is referenced as a high fraction of the positive control response, the procedure is seen to be uniformly more efficient than the fixed margin approach, yielding smaller sample sizes when sizing non-inferiority trials under identically specified conditions. Extending this method to proportions is straightforward, but highlights special considerations in the design of non-inferiority trials versus superiority trials, including potential trade-offs in statistical efficiency and interpretability.     

Non-inferiority trials: design concepts and issues - the encounters of academic consultants in statistics

Placebo-controlled trials are the ideal for evaluating medical treatment efficacy. They allow for control of the placebo effect and are most efficient, requiring the smallest numbers of patients to detect a treatment effect. A placebo control is ethically justified if no standard treatment exists, if the standard treatment has not been proven efficacious, there are no risks associated with delaying treatment or escape clauses are included in the protocol. Where possible and justified, they should be the first choice for medical treatment evaluation. Given the large number of proven effective treatments, placebo-controlled trials are often unethical. In these situations active-controlled trials are generally appropriate. The non-inferiority trial is appropriate for evaluation of the efficacy of an experimental treatment versus an active control when it is hypothesized that the experimental treatment may not be superior to a proven effective treatment, but is clinically and statistically not inferior in effectiveness. These trials are not easy to design. An active control must be selected. Good historical placebo-controlled trials documenting the efficacy of the active control must exist. From these historical trials statistical analysis must be performed and clinical judgement applied in order to determine the non-inferiority margin M and to assess assay sensitivity. The latter refers to establishing that the active drug would be superior to the placebo in the setting of the present non-inferiority trial (that is, the constancy assumption). Further, a putative placebo analysis of the new treatment versus the placebo using data from the non-inferiority trial and the historical active versus placebo-controlled trials is needed. Useable placebo-controlled historical trials for the active control are often not available, and determination of assay sensitivity and an appropriate M is difficult and debatable. Serious consideration to expansions of and alternatives to non-inferiority trials are needed.     

The use of putative placebo in active control trials: two applications in a regulatory setting

For life-threatening diseases, ethical considerations preclude the inclusion of an untreated control group in the investigation of a new therapeutic agent when a standard therapy exists. In these cases, active controlled studies are conducted, and may be planned to demonstrate either superiority or equivalence/non-inferiority of the new drug over the standard therapy (active control). In the non-inferiority study, an important aspect is the ability to detect an inferior drug (assay sensitivity). It has been suggested that assay sensitivity for a non-inferiority study should be deduced from historical data, specifically placebo controlled studies with the standard therapy. The assessment of assay sensitivity may also be important in a superiority trial that fails to demonstrate a statistically significant difference between treatments, and the sponsor attempts to determine whether there is lack of inferiority as an alternative hypothesis for regulatory approval. This paper describes two methods of putative placebo analysis for assessing assay sensitivity in active controlled trials. One approach imputes a point estimate for the odds ratio (95 per cent confidence interval) for a new drug (T) compared to a placebo control (P). A Bayesian approach calculates the posterior probability that T is superior to P, or, that T is at least k per cent as good as the active control (A) and A is more effective than P. These methods are applied in two clinical/regulatory settings: a phase III trial comparing docetaxel (Taxotere) to doxorubicin in metastatic breast cancer patients, and a phase III programme with two trials comparing enoxaparin (Lovenox) plus aspirin to unfractionated heparin plus aspirin in patients with unstable angina or non-Q-wave myocardial infarction. The methodologies presented in this paper were used in securing regulatory approval for docetaxel in the treatment of locally advanced or metastatic breast cancer after failure of prior chemotherapy, and for enoxaparin in the treatment of acute coronary syndrome.     

A comparison of intent-to-treat and per-protocol results in antibiotic non-inferiority trials

While the intent-to-treat (ITT) analysis is widely accepted for superiority trials, there remains debate about its role in non-inferiority trials. It is often said that the ITT tends to be anti-conservative in the demonstration of non-inferiority. This concern has led to some reliance on per-protocol (PP) analyses that exclude patients on the basis of post-baseline events, despite the inherent bias of such analyses. We compare ITT and PP results from antibiotic trials presented to the public at the FDA's Anti-infective Drug Advisory Committee from 1999 to 2003. While the number of available trials is too small to produce clear conclusions, these data did not support the assumption that the ITT would lead to smaller treatment difference than the PP, in the setting of antibiotic trials. Possible explanations are discussed.     

Choosing the analysis population in non-inferiority studies: per protocol or intent-to-treat

For superiority trials, the intent-to-treat population (ITT) is considered the primary analysis population because it tends to avoid the over-optimistic estimates of efficacy that results from a per-protocol (PP) population. However, the roles of the ITT population and PP population in non-inferiority studies are not clearly defined as in superiority trials. In this paper, a simulation study is conducted to systematically investigate the impact of different types of missingness and protocol violations on the conservatism or anticonservatism of analyses based on the ITT and the PP population in non-inferiority trials. We find that conservatism or anticonservatism of the PP or ITT analysis depends on many factors, including the type of protocol deviation and missingness, the treatment trajectory (for longitudinal study) and the method of handling missing data in ITT population. The requirement that non-inferiority be shown for both PP and ITT populations does not necessarily guarantee the validity of a non-inferiority conclusion and a sufficiently powered PP analysis is not necessarily powered for ITT analysis. It is important to assess the potential types and rates of protocol deviation and missingness that might occur in a non-inferiority trial and to obtain some prior knowledge regarding the treatment trajectory of the test treatment versus the active control at the design stage so that a proper analysis plan and appropriate power estimation can be carried out. In general, for the types of protocol violations and missingness considered, we find that hybrid ITT/PP analysis, which excludes non-compliant patients as in the PP analysis and properly addresses the impact of non-trivial missing data as in the MLE-based ITT analysis, is more promising by way of providing reliable non-inferiority tests.     

Non-inferiority test and confidence interval for the difference in correlated proportions in diagnostic procedures based on multiple raters

The efficacy of diagnostic procedures is generally evaluated on the basis of the results from multiple raters. However, there are few adequate methods of performing non-inferiority tests with confidence intervals to compare the accuracies (sensitivities or specificities) when multiple raters are considered. We propose new statistical methods for comparing the accuracies of two diagnostic procedures in a non-inferiority trial, on the basis of the results from multiple independent raters who are also independent of the study centers. We consider a study design in which each patient is subjected to two diagnostic procedures and all images are read by all raters. By assuming a multinomial distribution for matched-pair categorical data arising from the study design, we derive a score-based full menu, that is, a non-inferiority test, confidence interval and sample size formula, for inference of the difference in correlated proportions between the two diagnostic procedures. We conduct Monte Carlo simulation studies to examine the validity of the proposed methods, which showed that the proposed test has a size closer to the nominal significance level than a Wald-type test and that the proposed confidence interval has better empirical coverage probability than a Wald-type confidence interval. We illustrate the proposed methods with data from a study of diagnostic procedures for the diagnosis of oesophageal carcinoma infiltrating the tracheobronchial tree.     

Test non-inferiority (and equivalence) based on the odds ratio under a simple crossover trial

For testing the non-inferiority (or equivalence) of a generic drug to a standard drug, the odds ratio (OR) of patient response rates has been recommended to measure the relative treatment efficacy. On the basis of a random effects logistic regression model, we develop asymptotic test procedures for testing non-inferiority and equivalence with respect to the OR of patient response rates under a simple crossover design. We further derive exact test procedures, which are especially useful for the situations in which the number of patients in a crossover trial is small. We address sample size calculation for testing non-inferiority and equivalence based on the asymptotic test procedures proposed here. We also discuss estimation of the OR of patient response rates for both the treatment and period effects. Finally, we include two examples, one comparing two solution aerosols in treating asthma, and the other one studying two inhalation devices for asthmatics, to illustrate the use of the proposed test procedures and estimators.     

Pairwise non‐parametric non‐inferiority tests in 3 × 3 cross‐over trials: should we adjust for period?

Before an active compound can be registered as a drug, it needs to be demonstrated that it does not cause particular unwanted compound specific side effects. Demonstrating non-inferiority of an active drug with placebo with respect to an unwanted side effect is often based on pharmacodynamic 3 x 3 cross-over trials, with the third treatment arm consisting of a positive control. The main comparison then is the pairwise comparison between active drug and placebo. In this paper, two different non-parametric methods with adjustment for period effect are compared with the non-parametric non-adjusted test and the parametric period-adjusted test with respect to size and power. The non-parametric test with period adjustment based on rank alignment has generally the largest power, but its size exceeds the nominal significance level. The non-parametric test with period adjustment based on stratification has low power for trials with a small number of subjects. The non-parametric test without period adjustment is a valid alternative in such cases, but its power decreases substantially in the presence of period effects. In the case of unbalanced designs, however, only the non-parametric test with period adjustment based on stratification can be used.     

Testing superiority at interim analyses in a non-inferiority trial

Shift in research and development strategy from developing follow-on or 'me-too' drugs to differentiated medical products with potentially better efficacy than the standard of care (e.g., first-in-class, best-in-class, and bio-betters) highlights the scientific and commercial interests in establishing superiority even when a non-inferiority design, adequately powered for a pre-specified non-inferiority margin, is appropriate for various reasons. In this paper, we propose a group sequential design to test superiority at interim analyses in a non-inferiority trial. We will test superiority at the interim analyses using conventional group sequential methods, and we may stop the study because of better efficacy. If the study fails to establish superior efficacy at the interim and final analyses, we will test the primary non-inferiority hypothesis at the final analysis at the nominal level without alpha adjustment. Whereas superiority/non-inferiority testing no longer has the hierarchical structure in which the rejection region for testing superiority is a subset of that for testing non-inferiority, the impact of repeated superiority tests on the false positive rate and statistical power for the primary non-inferiority test at the final analysis is essentially ignorable. For the commonly used O'Brien-Fleming type alpha-spending function, we show that the impact is extremely small based upon Brownian motion boundary-crossing properties. Numerical evaluation further supports the conclusion for other alpha-spending functions with a substantial amount of alpha being spent on the interim superiority tests. We use a clinical trial example to illustrate the proposed design.