Opportunities and challenges of combined effect measures based on prioritized outcomes

Many authors have proposed different approaches to combine multiple endpoints in a univariate outcome measure in the literature. In case of binary or time‐to‐event variables, composite endpoints, which combine several event types within a single event or time‐to‐first‐event analysis are often used to assess the overall treatment effect. A main drawback of this approach is that the interpretation of the composite effect can be difficult as a negative effect in one component can be masked by a positive effect in another. Recently, some authors proposed more general approaches based on a priority ranking of outcomes, which moreover allow to combine outcome variables of different scale levels. These new combined effect measures assign a higher impact to more important endpoints, which is meant to simplify the interpretation of results. Whereas statistical tests and models for binary and time‐to‐event variables are well understood, the latter methods have not been investigated in detail so far. In this paper, we will investigate the statistical properties of prioritized combined outcome measures. We will perform a systematical comparison to standard composite measures, such as the all‐cause hazard ratio in case of time‐to‐event variables or the absolute rate difference in case of binary variables, to derive recommendations for different clinical trial scenarios. We will discuss extensions and modifications of the new effect measures, which simplify the clinical interpretation. Moreover, we propose a new method on how to combine the classical composite approach with a priority ranking of outcomes using a multiple testing strategy based on the closed test procedure. Copyright © 2013 John Wiley & Sons, Ltd.     

An approximate approach to sample size determination in bioequivalence testing with multiple pharmacokinetic responses

The approval of generic drugs requires the evidence of average bioequivalence (ABE) on both the area under the concentration–time curve and the peak concentration C_max. The bioequivalence (BE) hypothesis can be decomposed into the non‐inferiority (NI) and non‐superiority (NS) hypothesis. Most of regulatory agencies employ the two one‐sided tests (TOST) procedure to test ABE between two formulations. As it is based on the intersection–union principle, the TOST procedure is conservative in terms of the type I error rate. However, the type II error rate is the sum of the type II error rates with respect to each null hypothesis of NI and NS hypotheses. When the difference in population means between two treatments is not 0, no close‐form solution for the sample size for the BE hypothesis is available. Current methods provide the sample sizes with either insufficient power or unnecessarily excessive power. We suggest an approximate method for sample size determination, which can also provide the type II rate for each of NI and NS hypotheses. In addition, the proposed method is flexible to allow extension from one pharmacokinetic (PK) response to determination of the sample size required for multiple PK responses. We report the results of a numerical study. An R code is provided to calculate the sample size for BE testing based on the proposed methods. Copyright © 2014 John Wiley & Sons, Ltd.     

A new partition testing strategy for multiple endpoints

To evaluate efficacy in multiple endpoints in confirmatory clinical trials is a challenging problem in multiple hypotheses testing. The difficulty comes from the different importance of each endpoint and their underlying correlation. Current approaches to this problem, which test the efficacy in certain dose–endpoint combinations and collate the results, are based on closed testing or partition testing. Despite their different formulations, all current approaches test their dose–endpoint combinations as intersection hypotheses and apply various union‐intersection tests. Likelihood ratio test is seldom used owing to the extensive computation and lack of consistent inferences. In this article, we first generalize the formulation of multiple endpoints problem to include the cases of alternative primary endpoints and co‐primary endpoints. Then we propose a new partition testing approach that is based on consonance‐adjusted likelihood ratio test. The new procedure provides consistent inferences, and yet, it is still conservative and does not rely on the estimation of endpoint correlation or independence assumptions that might be challenged by regulatory agencies. Copyright © 2012 John Wiley & Sons, Ltd.     

基于相对度量指标的非劣效界值计算方法的比较

目的美国FDA在其非劣效临床试验指南中提供了两种基于相对度量指标的非劣效界值计算方法,一种是与绝对度量指标非劣效界值计算方法相对应的对数转换法(简称对数转换法),另外一种是直接基于相对风险的计算方法(简称直接计算法)。本文探讨了这两种方法计算结果的差异,以评估该差异对非劣效试验结果的影响。方法以风险比(hazard ratio,HR)为例,指代阳性对照药与安慰剂的疗效差异的保守估计值,从计算公式上阐述两种方法的关系,并展示两种方法在高优与低优指标中计算得到的非劣效界值的差异。结果当HR在0.80~1.25时,两种方法计算得到的非劣效界值差别在1%以内;当HR越远离1,差别越大。当取相同的效应保留比例f时,直接计算法计算出的非劣效界值总大于对数转换法,此时会导致低优指标使用直接计算法计算出的界值相对更激进,而高优指标则相对保守。当设定相同的非劣效界值δ时,对于低优指标,使用直接计算法所需设定的效应保留比例f高于对数转换法,对于高优指标则相反。结论在以相对度量指标作为主要评价指标的非劣效试验中,研究者应该认识到这两种计算方法对非劣效界值设定和试验结论的影响,综合考虑临床实践和试验药物的获益-风险等,慎重选取非劣效界值。     

Comparing vaccines: A systematic review of the use of the non-inferiority margin in vaccine trials

BackgroundNon-inferiority (NI) randomized controlled trials (RCTs) aim to demonstrate that a new treatment is no worse than a comparator that has already shown its efficacy over placebo within a pre-specified margin. However, clear guidelines on how the NI margin should be determined are lacking for vaccine trials. A difference (seroprevalence/risk) of 10% or a geometric mean titre/concentration (GMT) ratio of 1.5 or 2.0 in antibody levels is implicitly recommended for vaccine trials. We aimed to explore which NI margins were used in vaccine RCTs and how they were determined.MethodsA systematic search for NI vaccine RCTs yielded 177 eligible articles. Data were extracted from these articles using a standardized form and included general characteristics and characteristics specific for NI trials. Relations between the study characteristics and the NI margin used were explored.ResultsAmong the 143 studies using an NI margin based on difference (n = 136 on immunogenicity, n = 2 on efficacy and n = 5 on safety), 66% used a margin of 10%, 23% used margins lower than 10% (range 1–7.5%) and 11% used margins larger than 10% (range 11.5–25%). Of the 103 studies using a NI margin based on the GMT ratio, 50% used a margin of 0.67/1.5 and 49% used 0.5/2.0. As observed, 85% of the studies did not discuss the method of margin determination; and 19% of the studies lacked a confidence interval or p-value for non-inferiority.ConclusionMost NI vaccine RCTs used an NI margin of 10% for difference or a GMT ratio of 1.5 or 2.0 without a clear rationale. Most articles presented enough information for the reader to make a judgement about the NI margin used and the conclusions. The reporting on the design, margins used and results of NI vaccine trials could be improved; more explicit guidelines may help to achieve this end.     

A framework for considering the risk‐benefit trade‐off in designing noninferiority trials using composite outcome approaches

When a new treatment regimen is expected to have comparable or slightly worse efficacy to that of the control regimen but has benefits in other domains such as safety and tolerability, a noninferiority (NI) trial may be appropriate but is fraught with difficulty in justifying an acceptable NI margin that is based on both clinical and statistical input. To overcome this, we propose to utilize composite risk‐benefit outcomes that combine elements from domains of importance (eg, efficacy, safety, and tolerability). The composite outcome itself may be analyzed using a superiority framework, or it can be used as a tool at the design stage of a NI trial for selecting an NI margin for efficacy that balances changes in risks and benefits. In the latter case, the choice of NI margin may be based on a novel quantity called the maximum allowable decrease in efficacy (MADE), defined as the marginal difference in efficacy between arms that would yield a null treatment effect for the composite outcome given an assumed distribution for the composite outcome. We observe that MADE: (1) is larger when the safety improvement for the experimental arm is larger, (2) depends on the association between the efficacy and safety outcomes, and (3) depends on the control arm efficacy rate. We use a numerical example for power comparisons between a superiority test for the composite outcome vs a noninferiority test for efficacy using the MADE as the NI margin, and apply the methods to a TB treatment trial.     

Likelihood ratio and a Bayesian approach were superior to standard noninferiority analysis when the noninferiority margin varied with the control event rate

OBJECTIVE: To present and compare three statistical approaches for analyzing a noninferiority trial when the noninferiority margin depends on the control event rate. STUDY DESIGN AND SETTING: In noninferiority trials with a binary outcome, the noninferiority margin is often defined as a fixed delta, the largest clinically acceptable difference in event rates between treatment groups. An alternative and more flexible approach is to allow delta to vary according to the true event rate in the control group. The appropriate statistical method for evaluating noninferiority with a variable noninferiority margin is not apparent. Three statistical approaches are proposed and compared: an observed event rate (OER) approach based on equating the true control rate to the observed rate, a Bayesian approach, and a likelihood ratio test. RESULTS AND CONCLUSIONS: Simulations studies indicate that the proportion of trials in which noninferiority was erroneously demonstrated was higher for the OER approach than with the Bayesian and likelihood ratio approaches. In some cases, the Type I error rate exceeded 10% for the OER approach. The OER approach is not recommended for the analysis of noninferiority trials with a variable margin of equivalence. The Bayesian and likelihood ratio methods yielded better operating characteristics.     

About half of the noninferiority trials tested superior treatments: a trial-register based study

ObjectivesA concern that noninferiority (NI) trials pose a risk of degradation of the treatment effects is prevalent. Thus, we aimed to determine the fraction of positive true effects (superiority rate) and the average true effect of current NI trials based on data from registered NI trials.Study Design and SettingAll NI trials carried out between 2000 and 2007 analyzing the NI of efficacy as the primary objective and registered in one of the two major clinical trials registers were studied. Having retrieved results from these trials, random effects modeling of the effect estimates was performed to determine the distribution of true effects.ResultsEffect estimates were available for 79 of 99 eligible trials identified. For trials with binary outcome, we estimated a superiority rate of 49% (95% confidence interval = 27–70%) and a mean true log odds ratio of ?0.005 (?0.112, 0.102). For trials with continuous outcome, the superiority rate was 58% (41–74%) and the mean true effect as Cohen's d of 0.06 (?0.064, 0.192).ConclusionsThe unanticipated finding of a positive average true effect and superiority of the new treatment in most NI trials suggest that the current practice of choosing NI designs in clinical trials makes degradation on average unlikely. However, the distribution of true treatment effects demonstrates that, in some NI trials, the new treatment is distinctly inferior.     

Power and sample size computations in simultaneous tests for non-inferiority based on relative margins

In this paper, we address the problem of calculating power and sample sizes associated with simultaneous tests for non-inferiority. We consider the case of comparing several experimental treatments with an active control. The approach is based on the ratio view, where the common non-inferiority margin is chosen to be some percentage of the mean of the control treatment. Two power definitions in multiple hypothesis testing, namely, complete power and minimal power, are used in the computations. The sample sizes associated with the ratio-based inference are also compared with that of a comparable inference based on the difference of means for various scenarios. It is found that the sample size required for ratio-based inferences is smaller than that of difference-based inferences when the relative non-inferiority margin is less than one and when large response values indicate better treatment effects. The results are illustrated with examples.     

Bayesian methods for setting sample sizes and choosing allocation ratios in phase II clinical trials with time‐to‐event endpoints

Conventional phase II trials using binary endpoints as early indicators of a time‐to‐event outcome are not always feasible. Uveal melanoma has no reliable intermediate marker of efficacy. In pancreatic cancer and viral clearance, the time to the event of interest is short, making an early indicator unnecessary. In the latter application, Weibull models have been used to analyse corresponding time‐to‐event data. Bayesian sample size calculations are presented for single‐arm and randomised phase II trials assuming proportional hazards models for time‐to‐event endpoints. Special consideration is given to the case where survival times follow the Weibull distribution. The proposed methods are demonstrated through an illustrative trial based on uveal melanoma patient data. A procedure for prior specification based on knowledge or predictions of survival patterns is described. This enables investigation into the choice of allocation ratio in the randomised setting to assess whether a control arm is indeed required. The Bayesian framework enables sample sizes consistent with those used in practice to be obtained. When a confirmatory phase III trial will follow if suitable evidence of efficacy is identified, Bayesian approaches are less controversial than for definitive trials. In the randomised setting, a compromise for obtaining feasible sample sizes is a loss in certainty in the specified hypotheses: the Bayesian counterpart of power. However, this approach may still be preferable to running a single‐arm trial where no data is collected on the control treatment. This dilemma is present in most phase II trials, where resources are not sufficient to conduct a definitive trial. Copyright © 2015 John Wiley & Sons, Ltd.     

Accounting for informatively missing data in logistic regression by means of reassessment sampling

We explore the ‘reassessment’ design in a logistic regression setting, where a second wave of sampling is applied to recover a portion of the missing data on a binary exposure and/or outcome variable. We construct a joint likelihood function based on the original model of interest and a model for the missing data mechanism, with emphasis on non‐ignorable missingness. The estimation is carried out by numerical maximization of the joint likelihood function with close approximation of the accompanying Hessian matrix, using sharable programs that take advantage of general optimization routines in standard software. We show how likelihood ratio tests can be used for model selection and how they facilitate direct hypothesis testing for whether missingness is at random. Examples and simulations are presented to demonstrate the performance of the proposed method. Copyright © 2015 John Wiley & Sons, Ltd.     

Investigations on non‐inferiority—the Food and Drug Administration draft guidance on treatments for nosocomial pneumonia as a case for exact tests for binomial proportions

This paper addresses statistical issues in non‐inferiority trials where the primary outcome is a fatal event. The investigations are inspired by a recent Food and Drug Administration (FDA) draft guideline on treatments for nosocomial pneumonia. The non‐inferiority margin suggested in this guideline for the endpoint all‐cause mortality is defined on different distance measures (rate difference and odds ratio) and is discontinuous. Furthermore, the margin enables considerable power for the statistical proof of non‐inferiority at alternatives that might be regarded as clinically unacceptable, that is, even if the experimental treatment is harmful as compared with the control. We investigated the appropriateness of the proposed non‐inferiority margin as well as the performance of possible test statistics to be used for the analysis. A continuous variant of the margin proposed in the FDA guideline together with the unconditional exact test according to Barnard showed favorable characteristics with respect to type I error rate control and power. To prevent harmful new treatments from being declared as non‐inferior, we propose to add a ‘second hurdle’. We discuss examples and explore power characteristics when requiring both statistical significance and overcoming the second hurdle. Copyright © 2012 John Wiley & Sons, Ltd.     

Design and analysis of non-inferiority mortality trials in oncology

The recent revision of the Declaration of Helsinki and the existence of many new therapies that affect survival or serious morbidity, and that therefore cannot be denied patients, have generated increased interest in active-control trials, particularly those intended to show equivalence or non-inferiority to the active-control. A non-inferiority hypothesis has historically been formulated in terms of a fixed margin. This margin was historically designed to exclude a 'clinically meaningful difference', but has become recognized that the margin must also be no larger than the assured effect of the control in the new study. Depending on how this 'assured effect' is determined or estimated, the selected margin may be very small, leading to very large sample sizes, especially when there is an added requirement that a loss of some specified fraction of the assured effect must be ruled out. In cases where it is appropriate, this paper proposes non-inferiority analyses that do not involve a fixed margin, but can be described as a two confidence interval procedure that compares the 95 per cent two-sided CI for the difference between the treatment and the control to a confidence interval for the control effect (based on a meta-analysis of historical data comparing the control to placebo) that is chosen to preserve a study-wide type I error rate of about 0.025 (similar to the usual standard for a superiority trial) for testing for retention of a prespecified fraction of the control effect. The approach assumes that the estimate of the historical active-control effect size is applicable in the current study. If there is reason to believe that this effect size is diminished (for example, improved concomitant therapies) the estimate of this historical effect could be reduced appropriately. The statistical methodology for testing this non-inferiority hypothesis is developed for a hazard ratio (rather than an absolute difference between treatments, because a hazard ratio seems likely to be less population dependent than the absolute difference). In the case of oncology, the hazard ratio is the usual way of comparing treatments with respect to time to event (time to progression or survival) endpoints. The proportional hazards assumption is regarded as reasonable (approximately holding). The testing procedures proposed are conditionally equivalent to two confidence interval procedures that relax the conservatism of two 95 per cent confidence interval testing procedures and preserve the type I error rate at a one-sided 0.025 level. An application of this methodology to Xeloda, a recently approved drug for the treatment of metastatic colorectal cancers, is illustrated. Other methodologies are also described and assessed - including a point estimate procedure, a Bayesian procedure and two delta-method confidence interval procedures. Published in 2003 by John Wiley & Sons, Ltd.     

Statistical power of multiplicity adjustment strategies for correlated binary endpoints

There are numerous alternatives to the so-called Bonferroni adjustment to control for familywise Type I error among multiple tests. Yet, for the most part, these approaches disregard the correlation among endpoints. This can prove to be a conservative hypothesis testing strategy if the null hypothesis is false. The James procedure was proposed to account for the correlation structure among multiple continuous endpoints. Here, a simulation study evaluates the statistical power of the Hochberg and James adjustment strategies relative to that of the Bonferroni approach when used for multiple correlated binary variables. The simulations demonstrate that relative to the Bonferroni approach, neither alternative sacrifices power. The Hochberg approach has more statistical power for rho相似文献   

Weighted multiple testing correction for correlated tests

Virtually all clinical trials collect multiple endpoints that are usually correlated. Many methods have been proposed to control the family-wise type I error rate (FWER), but these methods often disregard the correlation among the endpoints, such as the commonly used Bonferroni correction, Holm procedure, Wiens' Bonferroni fixed-sequence (BFS) procedure and its extension, and the alpha-exhaustive fallback (AEF). Huque and Alosh proposed a flexible fixed-sequence (FFS) testing method, which extended the BFS method by taking into account correlations among endpoints. However, the FFS method faces a computational difficulty when there are four or more endpoints. Similar to the BFS procedure, the FFS method requires the prespecified testing sequence and the type I error rate used for first endpoint in the sequence (usually the most important endpoint) cannot be adjusted for the correlation among the endpoints or from the rejection of other null hypotheses for other endpoints. Thus, the power for this test is not maximized. In this paper, I present a weighted multiple testing correction for correlated tests. By using the package 'mvtnorm' in R, the proposed method can handle up to a thousand endpoints. Simulations show that the proposed method shares the advantage of the FFS and the AEF methods (having high power for the second or later hypotheses in the testing sequence) and has higher power for testing the first hypothesis than the FFS and the AEF methods. The proposed method has higher power for each individual hypothesis than the weighted Holm procedure, especially when the correlation between endpoints is high.     

Active-control trials with binary data: a comparison of methods for testing superiority or non-inferiority using the odds ratio

This paper considers methods for testing for superiority or non-inferiority in active-control trials with binary data, when the relative treatment effect is expressed as an odds ratio. Three asymptotic tests for the log-odds ratio based on the unconditional binary likelihood are presented, namely the likelihood ratio, Wald and score tests. All three tests can be implemented straightforwardly in standard statistical software packages, as can the corresponding confidence intervals. Simulations indicate that the three alternatives are similar in terms of the Type I error, with values close to the nominal level. However, when the non-inferiority margin becomes large, the score test slightly exceeds the nominal level. In general, the highest power is obtained from the score test, although all three tests are similar and the observed differences in power are not of practical importance.     

Mediation analysis for common binary outcomes

Mediation analysis provides an attractive causal inference framework to decompose the total effect of an exposure on an outcome into natural direct effects and natural indirect effects acting through a mediator. For binary outcomes, mediation analysis methods have been developed using logistic regression when the binary outcome is rare. These methods will not hold in practice when a disease is common. In this paper, we develop mediation analysis methods that relax the rare disease assumption when using logistic regression. We calculate the natural direct and indirect effects for common diseases by exploiting the relationship between logit and probit models. Specifically, we derive closed-form expressions for the natural direct and indirect effects on the odds ratio scale. Mediation models for both continuous and binary mediators are considered. We demonstrate through simulation that the proposed method performs well for common binary outcomes. We apply the proposed methods to analyze the Normative Aging Study to identify DNA methylation sites that are mediators of smoking behavior on the outcome of obstructed airway function.     

Point estimation and p‐values in phase II adaptive two‐stage designs with a binary endpoint

Clinical trials in phase II of drug development are frequently conducted as single‐arm two‐stage studies with a binary endpoint. Recently, adaptive designs have been proposed for this setting that enable a midcourse modification of the sample size. While these designs are elaborated with respect to hypothesis testing by assuring control of the type I error rate, the topic of point estimation has up to now not been addressed. For adaptive designs with a prespecified sample size recalculation rule, we propose a new point estimator that both assures compatibility of estimation and test decision and minimizes average mean squared error. This estimator can be interpreted as a constrained posterior mean estimate based on the non‐informative Jeffreys prior. A comparative investigation of the operating characteristics demonstrates the favorable properties of the proposed approach. Copyright © 2016 John Wiley & Sons, Ltd.     

A repeated measures model for analysis of continuous outcomes in sequential parallel comparison design studies

Previous authors have proposed the sequential parallel comparison design (SPCD) to address the issue of high placebo response rate in clinical trials. The original use of SPCD focused on binary outcomes, but recent use has since been extended to continuous outcomes that arise more naturally in many fields, including psychiatry. Analytic methods proposed to date for analysis of SPCD trial continuous data included methods based on seemingly unrelated regression and ordinary least squares. Here, we propose a repeated measures linear model that uses all outcome data collected in the trial and accounts for data that are missing at random. An appropriate contrast formulated after the model has been fit can be used to test the primary hypothesis of no difference in treatment effects between study arms. Our extensive simulations show that when compared with the other methods, our approach preserves the type I error even for small sample sizes and offers adequate power and the smallest mean squared error under a wide variety of assumptions. We recommend consideration of our approach for analysis of data coming from SPCD trials. Copyright © 2013 John Wiley & Sons, Ltd.     

A unifying approach for surrogate marker validation based on Prentice's criteria

Part of the recent literature on the evaluation of surrogate endpoints starts from a multi-trial approach which leads to a definition of validity in terms of the quality of both trial-level and individual-level association between a potential surrogate and a true endpoint, Buyse et al. These authors proposed their methodology based on the simplest cross-sectional case in which both the surrogate and the true endpoint are continuous and normally distributed. Different variations to this theme have been implemented for binary responses, times to event, combinations of binary and continuous endpoints, etc. However, a drawback of this methodology is that different settings have led to different definitions to quantify the association at the individual-level. In the longitudinal setting; Alonso et al. defined a class of canonical correlation functions that can be used to study surrogacy at the trial and individual-level. In the present work, we propose a new approach to evaluate surrogacy in the repeated measurements framework, we also show the connection between this proposal and the previous ones reported in the literature. Finally, we extend this concept to the non-normal case using the so-called 'likelihood reduction factor' (LRF) a new validation measure based on some of the Prentice's criteria. We apply the previous methodology using data from two clinical studies in psychiatry and ophthalmology.