Assessing overall evidence from noninferiority trials with shared historical data

For regulatory approval of a new drug, the United States Code of Federal Regulations (CFR) requires ‘substantial evidence’ from ‘adequate and well‐controlled investigations’. This requirement is interpreted in the Food and Drug Administration guidance as the need of ‘at least two adequate and well‐controlled studies, each convincing on its own to establish effectiveness’. The guidance also emphasizes the need of ‘independent substantiation of experimental results from multiple studies’. However, several authors have noted the loss of independence between two noninferiority trials that use the same set of historical data to make inferences, raising questions about whether the CFR requirement is met in noninferiority trials through current practice. In this article, we first propose a statistical interpretation of the CFR requirement in terms of trial‐level and overall type I error rates, which captures the essence of the requirement and can be operationalized for noninferiority trials. We next examine four typical regulatory settings in which the proposed requirement may or may not be fulfilled by existing methods of analysis (fixed margin and synthesis). In situations where the criteria are not met, we then propose adjustments to the existing methods. As illustrated with several examples, our results and findings can be helpful in designing and analyzing noninferiority trials in a way that is both compliant with the regulatory interpretation of the CFR requirement and reasonably powerful. Copyright © 2012 John Wiley & Sons, Ltd.