TACT method for non-inferiority testing in active controlled trials

In active controlled trials without a placebo arm, non-inferiority testing is often considered but has different objectives. For the objective of demonstrating the efficacy of an experimental treatment or retention of a fraction of the control effect by the treatment, there are two types of statistical methods for testing - the synthesis method and the confidence interval method. According to the study of Wang, Hung and Tsong, the former is efficient under the so-called constancy condition but may have the alpha error rate inflate rapidly if the condition does not hold. In contrast, the latter method with careful selection of the non-inferiority margin tends to be conservative if the condition holds and may still have a valid alpha error otherwise unless the effect of the active control is less to a large extent in the active controlled trial than in the historical trials. We developed the TACT method, Two-stage Active Control Testing, as a viable compromise between the two methods. Through the TACT method, the uninterpretable non-inferiority testing may be avoided prior to the end of the trial. The TACT method carefully constructed can have a valid alpha error rate and the power close to the synthesis method if the constancy condition holds. In addition, the TACT method is more powerful than the confidence interval method for testing for the efficacy of the new treatment relative to the putative placebo and for showing that the new treatment is not inferior to the active control comparator.     

Choice of lambda-margin and dependency of non-inferiority trials

For a two-arm active control clinical trial designed to test for non-inferiority of the test treatment compared with the active control standard treatment, data of historical studies are often utilized. For example, with a cross-trial comparison approach (also called synthetic approach or lambda-margin approach), the trial is conducted to test the hypothesis that the mean difference or the ratio between the current test product and the active control is no larger than a certain portion of the mean difference or the ratio of the active control and placebo obtained in the historical data when the positive response indicates treatment effectiveness. The regulatory agency usually requires that the clinical trials of two different test treatments are independent in most regular cases. It also requires, in general, two independent trials of the same test treatment in order to provide confirmatory evidence of the efficacy of the test product. In this article, we derived the relationship between the correlation of the test statistics of two trials with the choice of lambda (the percentage to preserve), the sample sizes and variances under the normality assumption. We showed that the smaller a lambda, the higher the correlation between the two non-inferiority tests. It is further shown that when an 80 per cent or larger lambda is used, the correlation can be controlled to be less than 10 per cent if the variances of the response variables in the current trial are not much smaller than those of the historical studies.     

Planning and analysis of three-arm non-inferiority trials with binary endpoints

Three-arm trials including an experimental treatment, an active control and a placebo group are frequently preferred for the assessment of non-inferiority. In contrast to two-arm non-inferiority studies, these designs allow a direct proof of efficacy of a new treatment by comparison with placebo. As a further advantage, the test problem for establishing non-inferiority can be formulated in such a way that rejection of the null hypothesis assures that a pre-defined portion of the (unknown) effect the reference shows versus placebo is preserved by the treatment under investigation. We present statistical methods for this study design and the situation of a binary outcome variable. Asymptotic test procedures are given and their actual type I error rates are calculated. Approximate sample size formulae are derived and their accuracy is discussed. Furthermore, the question of optimal allocation of the total sample size is considered. Power properties of the testing strategy including a pre-test for assay sensitivity are presented. The derived methods are illustrated by application to a clinical trial in depression.     

Choosing the analysis population in non-inferiority studies: per protocol or intent-to-treat

For superiority trials, the intent-to-treat population (ITT) is considered the primary analysis population because it tends to avoid the over-optimistic estimates of efficacy that results from a per-protocol (PP) population. However, the roles of the ITT population and PP population in non-inferiority studies are not clearly defined as in superiority trials. In this paper, a simulation study is conducted to systematically investigate the impact of different types of missingness and protocol violations on the conservatism or anticonservatism of analyses based on the ITT and the PP population in non-inferiority trials. We find that conservatism or anticonservatism of the PP or ITT analysis depends on many factors, including the type of protocol deviation and missingness, the treatment trajectory (for longitudinal study) and the method of handling missing data in ITT population. The requirement that non-inferiority be shown for both PP and ITT populations does not necessarily guarantee the validity of a non-inferiority conclusion and a sufficiently powered PP analysis is not necessarily powered for ITT analysis. It is important to assess the potential types and rates of protocol deviation and missingness that might occur in a non-inferiority trial and to obtain some prior knowledge regarding the treatment trajectory of the test treatment versus the active control at the design stage so that a proper analysis plan and appropriate power estimation can be carried out. In general, for the types of protocol violations and missingness considered, we find that hybrid ITT/PP analysis, which excludes non-compliant patients as in the PP analysis and properly addresses the impact of non-trivial missing data as in the MLE-based ITT analysis, is more promising by way of providing reliable non-inferiority tests.     

Assessing non-inferiority: a combination approach

Non-inferiority designs are growing in importance as a strategy for comparing new drugs with established therapies. Because it is not possible to show that a new drug and the established therapy have identical efficacy profiles, non-inferiority trials are designed to demonstrate that the new drug is not inferior to an established drug (the 'control') relative to a prespecified 'non-inferiority margin'. No objective principle guides the choice of the non-inferiority margin, and controversies about the margin have, in some cases, had important consequences for drug development.We argue that some of these controversies have arisen because non-inferiority trials must achieve two objectives. They must demonstrate not only that the new drug is not inferior to the control drug by the non-inferiority margin, but also that the new drug is superior to placebo. When the second objective is not considered explicitly, it can distort the choice of the non-inferiority margin. Some methods designed to address both objectives through the choice of the non-inferiority margin lead to overly stringent non-inferiority criteria.We describe an approach to non-inferiority analysis that combines two tests, a traditional test for non-inferiority and a test for superiority based on a synthetic estimate of the effect of the new treatment relative to placebo. The synthetic estimate may be 'discounted' to address concerns about assay inconstancy. We discuss power and sample size considerations for the proposed procedure.     

Two-stage adaptive strategy for superiority and non-inferiority hypotheses in active controlled clinical trials

In active controlled trials without a placebo arm, there are usually two study objectives: to test a superiority hypothesis that the experimental treatment is more effective than the active control therapy, and to test a non-inferiority hypothesis that the experimental treatment is therapeutically no worse than the active control within a defined margin. For a two-stage adaptive design, it is not necessary to give a fixed sample size calculation at the planning stage of the study when treatment effect information is often insufficient. Instead, decision and estimation of the design specifications can be made more reliably after the first stage when interim results are available. We propose the use of conditional power approach to determine the sample size and critical values for testing the superiority and non-inferiority hypotheses for the second stage based on the observed result of the first stage. The proposed adaptive procedure preserves the overall type I error rate for both superiority and non-inferiority, and has the flexibility of early termination of the study (for futility or efficacy) or extending the study by appropriate sample size.     

Improving the reporting of non-inferiority trials by incorporating non-efficacy benefits: not all non-inferiority trials are created equal

Non-inferiority trials are used to test if a novel intervention is not worse than a standard treatment by more than a prespecified amount, the non-inferiority margin (ΔNI). The ΔNI indicates the amount of efficacy loss in the primary outcome that is acceptable in exchange for non-efficacy benefits in other outcomes. However, non-inferiority designs are sometimes used when non-efficacy benefits are absent. Without non-efficacy benefits, loss in efficacy cannot be easily justified. Further, non-efficacy benefits are scarcely defined or considered by trialists when determining the magnitude of and providing justification for the non-inferiority margin. This is problematic as the importance of a treatment’s non-efficacy benefits are critical to understanding the results of a non-inferiority study. Here we propose the routine reporting in non-inferiority trial protocols and publications of non-efficacy benefits of the novel intervention along with the reporting of non-inferiority margins and their justification. The justification should include the specific trade-off between the accepted loss in efficacy (ΔNI) and the non-efficacy benefits of the novel treatment and should describe whether patients and other relevant stakeholders were involved in the definition of the ΔNI.

     

The use of putative placebo in active control trials: two applications in a regulatory setting

For life-threatening diseases, ethical considerations preclude the inclusion of an untreated control group in the investigation of a new therapeutic agent when a standard therapy exists. In these cases, active controlled studies are conducted, and may be planned to demonstrate either superiority or equivalence/non-inferiority of the new drug over the standard therapy (active control). In the non-inferiority study, an important aspect is the ability to detect an inferior drug (assay sensitivity). It has been suggested that assay sensitivity for a non-inferiority study should be deduced from historical data, specifically placebo controlled studies with the standard therapy. The assessment of assay sensitivity may also be important in a superiority trial that fails to demonstrate a statistically significant difference between treatments, and the sponsor attempts to determine whether there is lack of inferiority as an alternative hypothesis for regulatory approval. This paper describes two methods of putative placebo analysis for assessing assay sensitivity in active controlled trials. One approach imputes a point estimate for the odds ratio (95 per cent confidence interval) for a new drug (T) compared to a placebo control (P). A Bayesian approach calculates the posterior probability that T is superior to P, or, that T is at least k per cent as good as the active control (A) and A is more effective than P. These methods are applied in two clinical/regulatory settings: a phase III trial comparing docetaxel (Taxotere) to doxorubicin in metastatic breast cancer patients, and a phase III programme with two trials comparing enoxaparin (Lovenox) plus aspirin to unfractionated heparin plus aspirin in patients with unstable angina or non-Q-wave myocardial infarction. The methodologies presented in this paper were used in securing regulatory approval for docetaxel in the treatment of locally advanced or metastatic breast cancer after failure of prior chemotherapy, and for enoxaparin in the treatment of acute coronary syndrome.     

On the three-arm non-inferiority trial including a placebo with a prespecified margin

Three-arm trials including the experimental treatment, an active reference treatment and a placebo are recommended in the guidelines of the ICH and EMEA/CPMP as a useful approach to the assessment of assay sensitivity. Generally, the acceptable non-inferiority margin Δ has been defined as the maximum clinically irrelevant difference between treatments in many two-arm non-inferiority trials. However, many recent articles discussing three-arm trials have considered a design with unknown Δ which is the prespecified fraction f of unknown effect size of the reference drug, where the prespecified fraction f is treated as if it were a revised margin. Therefore, these methods cannot be applied to the case where the acceptable non-inferiority margin must be a prespecified difference between treatments. In this paper, we propose a statistical test procedure for three-arm non-inferiority trials with the margin Δ defined as a prespecified difference between treatments under the situation that the primary endpoints are normally distributed with a common, but unknown, variance. In addition, we derive the optimal allocation that minimizes the total sample size. The proposed method is illustrated with data on a randomized controlled trial on major depressive disorder.     

A two-stage sample size recalculation procedure for placebo- and active-controlled non-inferiority trials

Many non-inferiority trials of a test treatment versus an active control may also, if ethical, incorporate a placebo arm. Inclusion of a placebo arm enables a direct assessment of assay sensitivity. It also allows construction of a non-inferiority test that avoids the problematic specification of an absolute non-inferiority margin, and instead evaluates whether the test treatment preserves a pre-specified proportion of the effect of the active control over placebo. We describe a two-stage procedure for sample size recalculation in such a setting that maintains the desired power more closely than a fixed sample approach when the magnitude of the effect of the active control differs from that anticipated. We derive an allocation rule for randomization under which the procedure preserves the type I error rate, and show that this coincides with that previously presented for optimal allocation of the sample size among the three treatment arms.     

Toward rapproachment in the placebo control debate. A calculated compromise of power

In an effort to minimize risk to participants, some investigators avoid using placebo controls in randomized controlled clinical trials (RCT) if an effective treatment is available. An unintended consequence of this approach is that substantially more participants remain acutely ill (i.e., nonresponders) throughout an active-comparator trial than a placebo-controlled trial. This is due to the increased sample size required to detect smaller differences between investigational and proven active agents. The objective of this article is to identify an RCT design that will minimize both the number assigned to placebo and the number of nonresponders. To do so, two aspects of clinical trial design are manipulated: choice of comparator and treatment allocation ratio. Several examples illustrate empirically that placebo-controlled trials that are designed to randomize twice as many participants to the investigational cell could appeal to potential study participants, clinical researchers, and Institutional Review Boards alike.     

A note on the conservativeness of the confidence interval approach for the selection of non-inferiority margin in the two-arm active-control trial

Compared with placebo-control clinical trials, the interpretation of efficacy results from active-control trials requires more caution. This is because efficacy results from such trials cannot be reliably interpreted without a thorough understanding of the efficacy evidence that formed the basis for the approval of the active control, especially when such drug efficacy is to be established on the basis of clinical evidence from the traditional two-arm active-control clinical equivalence studies as opposed to the multi-arm active control. This is because in addition to over-reliance on the quantification of a clinically irrelevant acceptable margin of inferiority from historical data, such interpretation also depends on cross-trial inference for demonstration of experimental drug effect. We provide a brief overview of some design issues with the traditional two-arm active-control clinical trial and discuss regulators' concern regarding Type I error rate control (with the two most popular methods for the quantification of the non-inferiority margin) in cross-trial demonstration of experimental drug effect. Simulation results are presented to show that the point estimate method provides adequate control of the Type I error rate with > or =75 per cent retention of known active-control effect and that the confidence interval approach is uniformly ultra-conservative. We also report (via a numerical example from real clinical trial data) a couple of potentially less stringent alternative approaches for establishing the non-inferiority of a test drug over a control, which have been used in the past to provide additional efficacy evidence in NDA submission.     

Issues in planning and interpreting active control equivalence studies

Active control equivalence studies, with the goal of demonstrating therapeutic equivalence between a new and an active control treatment, are becoming more widespread due to current therapies that reflect previous successes in the development of new treatments. Because ethical requirements preclude the use of a placebo or no-treatment control for internal study validation, certain methodologic issues arise in active control equivalence trials that require special attention. We emphasize a special feature of this alternative study design, namely, its reliance on an implicit "historical control assumption". To conclude that a new drug is efficacious on the basis of an active control equivalence study (ACES) requires a fundamental assumption that the active control drug would have performed better than a placebo, had a placebo been used in the trial. In designing an ACES, one needs some assurance that historical estimates of the active control drug's efficacy relative to placebo are applicable to the new experimental setting. Steps that can be taken to compile such evidence and to justify the use of an active control equivalence design are described. These issues are illustrated in the context of a planned study to evaluate the efficacy of a new drug for the prevention of stroke, using aspirin as an active control.     

Controlling the type 1 error rate in non-inferiority trials

Two different approaches have been proposed for establishing the efficacy of an experimental therapy through a non-inferiority trial: The fixed-margin approach involves first defining a non-inferiority margin and then demonstrating that the experimental therapy is not worse than the control by more than this amount, and the synthesis approach involves combining the data from the non-inferiority trial with the data from historical trials evaluating the effect of the control. In this paper, we introduce a unified approach that has both these approaches as special cases and show how the parameters of this approach can be selected to control the unconditional type 1 error rate in the presence of departures from the assumptions of assay sensitivity and constancy. It is shown that the fixed-margin approach can be extremely inefficient and that it is always possible to achieve equivalent control of the unconditional type 1 error rate, with higher power, by using an appropriately chosen synthesis method.     

Design and analysis of non-inferiority mortality trials in oncology

The recent revision of the Declaration of Helsinki and the existence of many new therapies that affect survival or serious morbidity, and that therefore cannot be denied patients, have generated increased interest in active-control trials, particularly those intended to show equivalence or non-inferiority to the active-control. A non-inferiority hypothesis has historically been formulated in terms of a fixed margin. This margin was historically designed to exclude a 'clinically meaningful difference', but has become recognized that the margin must also be no larger than the assured effect of the control in the new study. Depending on how this 'assured effect' is determined or estimated, the selected margin may be very small, leading to very large sample sizes, especially when there is an added requirement that a loss of some specified fraction of the assured effect must be ruled out. In cases where it is appropriate, this paper proposes non-inferiority analyses that do not involve a fixed margin, but can be described as a two confidence interval procedure that compares the 95 per cent two-sided CI for the difference between the treatment and the control to a confidence interval for the control effect (based on a meta-analysis of historical data comparing the control to placebo) that is chosen to preserve a study-wide type I error rate of about 0.025 (similar to the usual standard for a superiority trial) for testing for retention of a prespecified fraction of the control effect. The approach assumes that the estimate of the historical active-control effect size is applicable in the current study. If there is reason to believe that this effect size is diminished (for example, improved concomitant therapies) the estimate of this historical effect could be reduced appropriately. The statistical methodology for testing this non-inferiority hypothesis is developed for a hazard ratio (rather than an absolute difference between treatments, because a hazard ratio seems likely to be less population dependent than the absolute difference). In the case of oncology, the hazard ratio is the usual way of comparing treatments with respect to time to event (time to progression or survival) endpoints. The proportional hazards assumption is regarded as reasonable (approximately holding). The testing procedures proposed are conditionally equivalent to two confidence interval procedures that relax the conservatism of two 95 per cent confidence interval testing procedures and preserve the type I error rate at a one-sided 0.025 level. An application of this methodology to Xeloda, a recently approved drug for the treatment of metastatic colorectal cancers, is illustrated. Other methodologies are also described and assessed - including a point estimate procedure, a Bayesian procedure and two delta-method confidence interval procedures. Published in 2003 by John Wiley & Sons, Ltd.     

Some fundamental issues with non-inferiority testing in active controlled trials

In an active controlled non-inferiority trial without a placebo arm, it is often not entirely clear what the primary objective is. In many cases the considered goal is to demonstrate that the experimental treatment preserves at least some fraction of the effect of the active control. The active control effect is a parameter, the value of which is unknown. To test the hypothesis of effect preservation, the classical confidence interval approach requires specification of a non-inferiority margin which is a function of the unknown active control effect. When the margin is estimated, it is also not clear what is the relevant type I error of making a false assertion about preservation of the active control effect. The statistical uncertainty of the estimated margin arguably needs to be incorporated in evaluation of the type I error. In this paper we discuss these fundamental issues. We show that the classical confidence interval approach cannot attain the target type I error exactly since this error varies as the sample size or as the values of the nuisance parameters in the active controlled trial change. In contrast, the preservation tests, as proposed in literature, can attain the target type I error rate exactly, regardless of the sample size and the values of the nuisance parameters, but can do so only at the price of several strong assumptions holding that may not be directly verifiable. One assumption is the constancy condition holding whereby the effect of the active control in the historical trial populations is assumed to carry to the population of the active control trial. When this condition is violated, both the confidence interval approach and the preservation test method may be problematic.     

On non-inferiority margin and statistical tests in active control trials

The problem of selecting a non-inferiority margin and the corresponding statistical test for non-inferiority in active control trials is considered. For selection of non-inferiority margin, the guideline by the International Conference on Harmonization (ICH) recommends that the non-inferiority margin should be chosen in such a way that if the non-inferiority of the test therapy to the active control agent is claimed, the test therapy is not only non-inferior to the active control agent, but also superior to the placebo. Furthermore, variability should be taken into account. Along this line, a method for selecting non-inferiority margins with some statistical justification is proposed. Statistical tests for non-inferiority designed in the situation where the non-inferiority margin is an unknown parameter are derived. An example concerning a cancer trail for testing non-inferiority with the primary study endpoint of the time to disease progression is presented to illustrate the proposed method.     

Bayesian inference for a principal stratum estimand to assess the treatment effect in a subgroup characterized by postrandomization event occurrence

The treatment effect in subgroups of patients is often of interest in randomized controlled clinical trials, as this may provide useful information on how to treat which patients best. When a specific subgroup is characterized by the absence of certain events that happen postrandomization, a naive analysis on the subset of patients without these events may be misleading. The principal stratification framework allows one to define an appropriate causal estimand in such settings. Statistical inference for the principal stratum estimand hinges on scientifically justified assumptions, which can be included with Bayesian methods through prior distributions. Our motivating example is a large randomized placebo-controlled trial of siponimod in patients with secondary progressive multiple sclerosis. The primary objective of this trial was to demonstrate the efficacy of siponimod relative to placebo in delaying disability progression for the whole study population. However, the treatment effect in the subgroup of patients who would not relapse during the trial is relevant from both a scientific and patient perspective. Assessing this subgroup treatment effect is challenging as there is strong evidence that siponimod reduces relapses. We describe in detail the scientific question of interest, the principal stratum estimand, the corresponding analysis method for binary endpoints, and sensitivity analyses. Although our work is motivated by a randomized clinical trial, the approach has broader appeal and could be adapted for observational studies.     

The European regulatory experience

This paper discusses four methodological topics that have been a regular source of difficulty and debate in European regulatory work. (i) The increasing use of non-inferiority trials in the development of medicinal products has highlighted several problems. These relate first to the choice of the non-inferiority margin and secondly to the circumstances under which a non-inferiority design is or is not appropriate. (ii) The use of meta-analysis in regulatory applications is still controversial and acceptable uses need to be defined. (iii) Analysis of responders provides a useful insight into the size of treatment benefits but can be misleading, especially when it is impossible to be certain whether or not an individual patient has truly responded to treatment. (iv) The extent of the monitoring of clinical trial procedures and data still distinguishes industry-sponsored trials from other trials: it is not clear that it should. These questions are all equally important for those involved in clinical trial work outside the arena of pharmaceutical development.     

Imputing gene-treatment interactions when the genotype distribution is unknown using case-only and putative placebo analyses--a new method for the Genetics of Hypertension Associated Treatment (GenHAT) study

There is a sizeable literature on methods for detecting gene-environment interaction in the framework of case-control studies, particularly with reference to the assumption of independence of genotype and exposure. In the context of a clinical trial, wherein gene-drug interactions with regard to outcomes are examined, these methods may be readily applied, as gene and drug are independent by randomization. In an active-controlled trial (experimental treatment vs standard) that has collected genotype information, gene-drug interactions can be estimated. In addition, the effect of the experimental treatment vs placebo can be imputed by using data from a historical placebo-controlled trial (standard vs placebo) if either (a) genotype information is available from the historical trial or (b) assumptions are made about the prevalence of genotype and the odds ratios of genotype and disease in the historical trial using information from other studies. Motivation for these procedures is provided by the Genetics of Hypertension Associated Treatment, a large pharmacogenetics, ancillary study of a hypertension clinical trial, and examples from published hypertension trials will be used to illustrate the methods.