Choosing the analysis population in non-inferiority studies: per protocol or intent-to-treat

For superiority trials, the intent-to-treat population (ITT) is considered the primary analysis population because it tends to avoid the over-optimistic estimates of efficacy that results from a per-protocol (PP) population. However, the roles of the ITT population and PP population in non-inferiority studies are not clearly defined as in superiority trials. In this paper, a simulation study is conducted to systematically investigate the impact of different types of missingness and protocol violations on the conservatism or anticonservatism of analyses based on the ITT and the PP population in non-inferiority trials. We find that conservatism or anticonservatism of the PP or ITT analysis depends on many factors, including the type of protocol deviation and missingness, the treatment trajectory (for longitudinal study) and the method of handling missing data in ITT population. The requirement that non-inferiority be shown for both PP and ITT populations does not necessarily guarantee the validity of a non-inferiority conclusion and a sufficiently powered PP analysis is not necessarily powered for ITT analysis. It is important to assess the potential types and rates of protocol deviation and missingness that might occur in a non-inferiority trial and to obtain some prior knowledge regarding the treatment trajectory of the test treatment versus the active control at the design stage so that a proper analysis plan and appropriate power estimation can be carried out. In general, for the types of protocol violations and missingness considered, we find that hybrid ITT/PP analysis, which excludes non-compliant patients as in the PP analysis and properly addresses the impact of non-trivial missing data as in the MLE-based ITT analysis, is more promising by way of providing reliable non-inferiority tests.     

A Bayesian sensitivity model for intention‐to‐treat analysis on binary outcomes with dropouts

Intention‐to‐treat (ITT) analysis is commonly used in randomized clinical trials. However, the use of ITT analysis presents a challenge: how to deal with subjects who drop out. Here we focus on randomized trials where the primary outcome is a binary endpoint. Several approaches are available for including the dropout subject in the ITT analysis, mainly chosen prior to unblinding the study. These approaches reduce the potential bias due to breaking the randomization code. However, the validity of the results will highly depend on untestable assumptions about the dropout mechanism. Thus, it is important to evaluate the sensitivity of the results across different missing‐data mechanisms. We propose here a Bayesian pattern‐mixture model for ITT analysis of binary outcomes with dropouts that applies over different types of missing‐data mechanisms. We introduce a new parameterization to identify the model, which is then used for sensitivity analysis. The parameterization is defined as the odds ratio of having an endpoint between the subjects who dropped out and those who completed the study. Such parameterization is intuitive and easy to use in sensitivity analysis; it also incorporates most of the available methods as special cases. The model is applied to TRial Of Preventing HYpertension. Copyright © 2008 John Wiley & Sons, Ltd.     

A graphical sensitivity analysis for clinical trials with non-ignorable missing binary outcome

Many clinical trials are analysed using an intention-to-treat (ITT) approach. A full application of the ITT approach is only possible when complete outcome data are available for all randomized subjects. In a recent survey of clinical trial reports including an ITT analysis, complete case analysis (excluding all patients with a missing response) was common. This does not comply with the basic principles of ITT since not all randomized subjects are included in the analysis. Analyses of data with missing values are based on untestable assumptions, and so sensitivity analysis presenting a range of estimates under alternative assumptions about the missing-data mechanism is recommended. For binary outcome, extreme case analysis has been suggested as a simple form of sensitivity analysis, but this is rarely conclusive. A graphical sensitivity analysis is proposed which displays the results of all possible allocations of cases with missing binary outcome. Extension to allow binomial variation in outcome is also considered. The display is based on easily interpretable parameters and allows informal examination of the effects of varying prior beliefs.     

Pragmatic randomised controlled trials in parenting research: the issue of intention to treat

STUDY OBJECTIVE: To evaluate trials of parenting programmes, regarding their use of intention to treat (ITT). DESIGN: Individual trials included in two relevant Cochrane systematic reviews were scrutinised by two independent reviewers. Data on country of origin, target audience, trial type, treatment violations, use of ITT, and the management of missing data were extracted. MAIN RESULTS: Thirty trial reports were reviewed. Three reported the use of an ITT approach to data analysis. Nineteen reported losing subjects to follow up although the implications of this were rarely considered. Insufficient detail in reports meant it was difficult to identify study drop outs, the nature of treatment violations, and those failing to provide outcome assessments. In two trials, study drop outs were considered as additional control groups, violating the basic principle of ITT. CONCLUSIONS: It is recommended that future trial reports adhere to CONSORT guidelines. In particular ITT should be used for the main analyses, with strategies for managing treatment violations and handling missing data being reported a priori. Those conducting trials need to acknowledge the social nature of these programmes can sometimes result in erratic parent attendance and participation, which would only increase the chances of missing data. The use of approaches that can limit the proportion of missing data is therefore recommended.     

Bayesian inference for randomized clinical trials with treatment failures

During the course of a clinical trial, subjects may experience treatment failure. For ethical reasons, it is necessary to administer emergency or rescue medications for such subjects. However, the rescue medications may bias the set of response measurements. This bias is of particular concern if a subject has been randomized to the control group, and the rescue medications improve the subject's condition. The standard approach to analysing data from a clinical trial is to perform an intent-to-treat (ITT) analysis, wherein the data are analysed according to treatment randomization. Supplementary analyses may be performed in addition to the ITT analysis to account for the effect of treatment failures and rescue medications. A Bayesian, counterfactual approach, which uses the data augmentation (DA) algorithm, is proposed for supplemental analysis. A simulation study is conducted to compare the operating characteristics of this procedure with a likelihood-based, counterfactual approach based on the EM algorithm. An example from the Asthma Clinical Research Network (ACRN) is used to illustrate the Bayesian procedure.     

Intention-to-treat approach to data from randomized controlled trials: a sensitivity analysis

The intention-to-treat (ITT) approach to randomized controlled trials analyzes data on the basis of treatment assignment, not treatment receipt. Alternative approaches make comparisons according to the treatment received at the end of the trial (as-treated analysis) or using only subjects who did not deviate from the assigned treatment (adherers-only analysis). Using a sensitivity analysis on data for a hypothetical trial, we compare these different analytical approaches in the context of two common protocol deviations: loss to follow-up and switching across treatments. In each case, two rates of deviation are considered: 10% and 30%. The analysis shows that biased estimates of effect may occur when deviation is nonrandom, when a large percentage of participants switch treatments or are lost to follow-up, and when the method of estimating missing values accounts inadequately for the process causing loss to follow-up. In general, ITT analysis attenuates between-group effects. Trialists should use sensitivity analyses on their data and should compare the characteristics of participants who do and those who do not deviate from the trial protocol. The ITT approach is not a remedy for unsound design, and imputation of missing values is not a substitute for complete, good quality data.     

A comparison of the results of intent‐to‐treat,per‐protocol,and g‐estimation in the presence of non‐random treatment changes in a time‐to‐event non‐inferiority trial

While intent‐to‐treat (ITT) analysis is widely accepted for superiority trials, there remains debate about its role in non‐inferiority trials. It has often been said that ITT analysis tends to be anti‐conservative in demonstrating non‐inferiority, suggesting that per‐protocol (PP) analysis may be preferable for non‐inferiority trials, despite the inherent bias of such analyses. We propose using randomization‐based g‐estimation analyses that more effectively preserve the integrity of randomization than do the more widely used PP analyses. Simulation studies were conducted to investigate the impacts of different types of treatment changes on the conservatism or anti‐conservatism of analyses using the ITT, PP, and g‐estimation methods in a time‐to‐event outcome. The ITT results were anti‐conservative for all simulations. Anti‐conservativeness increased with the percentage of treatment change and was more pronounced for outcome‐dependent treatment changes. PP analysis, in which treatment‐switching cases were censored at the time of treatment change, maintained type I error near the nominal level for independent treatment changes, whereas for outcome‐dependent cases, PP analysis was either conservative or anti‐conservative depending on the mechanism underlying the percentage of treatment changes. G‐estimation analysis maintained type I error near the nominal level even for outcome‐dependent treatment changes, although information on unmeasured covariates is not used in the analysis. Thus, randomization‐based g‐estimation analyses should be used to supplement the more conventional ITT and PP analyses, especially for non‐inferiority trials. Copyright © 2010 John Wiley & Sons, Ltd.     

Allowing for uncertainty due to missing data in meta-analysis--part 1: two-stage methods

Analysis of a randomized trial with missing outcome data involves untestable assumptions, such as the missing at random (MAR) assumption. Estimated treatment effects are potentially biased if these assumptions are wrong. We quantify the degree of departure from the MAR assumption by the informative missingness odds ratio (IMOR). We incorporate prior beliefs about the IMOR in a Bayesian pattern-mixture model and derive a point estimate and standard error that take account of the uncertainty about the IMOR. In meta-analysis, this model should be used for four separate sensitivity analyses which explore the impact of IMORs that either agree or contrast across trial arms on pooled results via their effects on point estimates or on standard errors. We also propose a variance inflation factor that can be used to assess the influence of trials with many missing outcomes on the meta-analysis. We illustrate the methods using a meta-analysis on psychiatric interventions in deliberate self-harm.     

Discordance between reported intention-to-treat and per protocol analyses

OBJECTIVE: To quantify the degree of disagreement between the two most popular methods for dealing with missing data: intention to treat (ITT) and per protocol (PP). STUDY DESIGN AND SETTING: We performed a systematic review of randomized two-armed clinical trials (CTs) published between 2001 and 2003, abstracted in PubMed and reporting both the ITT and PP analyses on a primary binary endpoint, out of which 74 papers were finally selected. The treatment effect of each CT was measured by the odds ratio, and the disagreement between them was quantified by the Bland-Altman method. RESULTS: On average, the PP estimator provides greater values Log(e)ORPP=1 x 25.Log(e)ORITT, (95% CI: 1.15, 1.35) than the corresponding ITT estimator, although the limits of concordance showed that the ratio between the two estimators varies greatly from 0.39 up to 2.53. CONCLUSION: These results confirm that missing values may cause both systematic and unpredictable bias in CTs. Further efforts should be made to minimize protocol deviations and to use better statistical methods to highlight the drawbacks of missing information. In the presence of protocol deviations, the conclusion of a CT cannot rest on the single reporting of either the ITT or the PP approach alone.     

Sensitivity analysis for a partially missing binary outcome in a two‐arm randomized clinical trial

Although recent guidelines for dealing with missing data emphasize the need for sensitivity analyses, and such analyses have a long history in statistics, universal recommendations for conducting and displaying these analyses are scarce. We propose graphical displays that help formalize and visualize the results of sensitivity analyses, building upon the idea of ‘tipping‐point’ analysis for randomized experiments with a binary outcome and a dichotomous treatment. The resulting ‘enhanced tipping‐point displays’ are convenient summaries of conclusions obtained from making different modeling assumptions about missingness mechanisms. The primary goal of the displays is to make formal sensitivity analysesmore comprehensible to practitioners, thereby helping them assess the robustness of the experiment's conclusions to plausible missingness mechanisms. We also present a recent example of these enhanced displays in amedical device clinical trial that helped lead to FDA approval. Copyright © 2014 John Wiley & Sons, Ltd.     

Current issues in non-inferiority trials

Non-inferiority (NI) trials enable a direct comparison of the relative benefit-to-risk profiles of an experimental intervention and a standard-of-care regimen. When the standard has clinical efficacy of substantial magnitude that is precisely estimated ideally using data from multiple adequate and well-controlled trials, with such estimates being relevant to the setting of the NI trial, then the NI trial can provide the scientific and regulatory evidence required to reliably assess the efficacy of the new intervention. In clinical practice, considerable uncertainty remains regarding when such trials should be conducted, how they should be designed, what standards for quality of trial conduct must be achieved, and how results should be interpreted. Recent examples will be considered to provide important insights and to highlight some of the challenges that remain to be adequately addressed regarding the use of the NI approach for the evaluation of new interventions. 'Imputed placebo' and 'margin'-based approaches to NI trial design will be considered, as well as the risk of 'bio-creep' with repeated NI trials, use of NI trials when determining whether excess safety risks can be ruled out, higher standards regarding quality of study conduct required with NI trials, and the myth that NI trials always require huge sample sizes.     

The effects of non-compliance on intent-to-treat analysis of equivalence trials

The standard approach for analysing a randomized clinical trial is based on intent-to-treat (ITT) where subjects are analysed according to their assigned treatment group regardless of actual adherence to the treatment protocol. For therapeutic equivalence trials, it is a common concern that an ITT analysis increases the chance of erroneously concluding equivalence. In this paper, we formally investigate the impact of non-compliance on an ITT analysis of equivalence trials with a binary outcome. We assume 'all-or-none' compliance and independence between compliance and the outcome. Our results indicate that non-compliance does not always make it easier to demonstrate equivalence. The direction and magnitude of changes in the type I error rate and power of the study depend on the patterns of non-compliance, event probabilities, the margin of equivalence and other factors.     

Correcting for non‐compliance in randomized non‐inferiority trials with active and placebo control using structural models

The three‐arm clinical trial design, which includes a test treatment, an active reference, and placebo control, is the gold standard for the assessment of non‐inferiority. In the presence of non‐compliance, one common concern is that an intent‐to‐treat (ITT) analysis (which is the standard approach to non‐inferiority trials), tends to increase the chances of erroneously concluding non‐inferiority, suggesting that the per‐protocol (PP) analysis may be preferable for non‐inferiority trials despite its inherent bias. The objective of this paper was to develop statistical methodology for dealing with non‐compliance in three‐arm non‐inferiority trials for censored, time‐to‐event data. Changes in treatment were here considered the only form of non‐compliance. An approach using a three‐arm rank preserving structural failure time model and G‐estimation analysis is here presented. Using simulations, the impact of non‐compliance on non‐inferiority trials was investigated in detail using ITT, PP analyses, and the present proposed method. Results indicate that the proposed method shows good characteristics, and that neither ITT nor PP analyses can always guarantee the validity of the non‐inferiority conclusion. A Statistical Analysis System program for the implementation of the proposed test procedure is available from the authors upon request. Copyright © 2014 John Wiley & Sons, Ltd.     

A comparison of intent-to-treat and per-protocol results in antibiotic non-inferiority trials

While the intent-to-treat (ITT) analysis is widely accepted for superiority trials, there remains debate about its role in non-inferiority trials. It is often said that the ITT tends to be anti-conservative in the demonstration of non-inferiority. This concern has led to some reliance on per-protocol (PP) analyses that exclude patients on the basis of post-baseline events, despite the inherent bias of such analyses. We compare ITT and PP results from antibiotic trials presented to the public at the FDA's Anti-infective Drug Advisory Committee from 1999 to 2003. While the number of available trials is too small to produce clear conclusions, these data did not support the assumption that the ITT would lead to smaller treatment difference than the PP, in the setting of antibiotic trials. Possible explanations are discussed.     

Reference‐based multiple imputation for missing data sensitivity analyses in trial‐based cost‐effectiveness analysis

Missing data are a common issue in cost‐effectiveness analysis (CEA) alongside randomised trials and are often addressed assuming the data are ‘missing at random’. However, this assumption is often questionable, and sensitivity analyses are required to assess the implications of departures from missing at random. Reference‐based multiple imputation provides an attractive approach for conducting such sensitivity analyses, because missing data assumptions are framed in an intuitive way by making reference to other trial arms. For example, a plausible not at random mechanism in a placebo‐controlled trial would be to assume that participants in the experimental arm who dropped out stop taking their treatment and have similar outcomes to those in the placebo arm. Drawing on the increasing use of this approach in other areas, this paper aims to extend and illustrate the reference‐based multiple imputation approach in CEA. It introduces the principles of reference‐based imputation and proposes an extension to the CEA context. The method is illustrated in the CEA of the CoBalT trial evaluating cognitive behavioural therapy for treatment‐resistant depression. Stata code is provided. We find that reference‐based multiple imputation provides a relevant and accessible framework for assessing the robustness of CEA conclusions to different missing data assumptions.     

Allowing for uncertainty due to missing data in meta-analysis--part 2: hierarchical models

We propose a hierarchical model for the analysis of data from several randomized trials where some outcomes are missing. The degree of departure from a missing-at-random assumption in each arm of each trial is expressed by an informative missing odds ratio (IMOR). We require a realistic prior for the IMORs, including an assessment of the prior correlation between IMORs in different arms and in different trials. The model is fitted by Monte Carlo Markov Chain techniques. By applying the method in three different data sets, we show that it is possible to appropriately capture the extra uncertainty due to missing data, and we discuss in what circumstances it is possible to learn about the IMOR.     

Allowing for uncertainty due to missing continuous outcome data in pairwise and network meta‐analysis

Missing outcome data are commonly encountered in randomized controlled trials and hence may need to be addressed in a meta‐analysis of multiple trials. A common and simple approach to deal with missing data is to restrict analysis to individuals for whom the outcome was obtained (complete case analysis). However, estimated treatment effects from complete case analyses are potentially biased if informative missing data are ignored. We develop methods for estimating meta‐analytic summary treatment effects for continuous outcomes in the presence of missing data for some of the individuals within the trials. We build on a method previously developed for binary outcomes, which quantifies the degree of departure from a missing at random assumption via the informative missingness odds ratio. Our new model quantifies the degree of departure from missing at random using either an informative missingness difference of means or an informative missingness ratio of means, both of which relate the mean value of the missing outcome data to that of the observed data. We propose estimating the treatment effects, adjusted for informative missingness, and their standard errors by a Taylor series approximation and by a Monte Carlo method. We apply the methodology to examples of both pairwise and network meta‐analysis with multi‐arm trials. © 2014 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.     

Bayesian estimation,simulation and uncertainty analysis: the cost-effectiveness of ganciclovir prophylaxis in liver transplantation

This paper demonstrates the usefulness of combining simulation with Bayesian estimation methods in analysis of cost-effectiveness data collected alongside a clinical trial. Specifically, we use Markov Chain Monte Carlo (MCMC) to estimate a system of generalized linear models relating costs and outcomes to a disease process affected by treatment under alternative therapies. The MCMC draws are used as parameters in simulations which yield inference about the relative cost-effectiveness of the novel therapy under a variety of scenarios. Total parametric uncertainty is assessed directly by examining the joint distribution of simulated average incremental cost and effectiveness. The approach allows flexibility in assessing treatment in various counterfactual premises and quantifies the global effect of parametric uncertainty on a decision-maker's confidence in adopting one therapy over the other.     

Meta-analysis of multiple outcomes by regression with random effects

Earlier work showed how to perform fixed-effects meta-analysis of studies or trials when each provides results on more than one outcome per patient and these multiple outcomes are correlated. That fixed-effects generalized-least-squares approach analyzes the multiple outcomes jointly within a single model, and it can include covariates, such as duration of therapy or quality of trial, that may explain observed heterogeneity of results among the trials. Sometimes the covariates explain all the heterogeneity, and the fixed-effects regression model is appropriate. However, unexplained heterogeneity may often remain, even after taking into account known or suspected covariates. Because fixed-effects models do not make allowance for this remaining unexplained heterogeneity, the potential exists for bias in estimated coefficients, standard errors and p-values. We propose two random-effects approaches for the regression meta-analysis of multiple correlated outcomes. We compare their use with fixed-effects models and with separate-outcomes models in a meta-analysis of periodontal clinical trials. A simulation study shows the advantages of the random-effects approach. These methods also facilitate meta-analysis of trials that compare more than two treatments. © 1998 John Wiley & Sons, Ltd.     

A Bayesian design for phase II clinical trials with delayed responses based on multiple imputation

Interim monitoring is routinely conducted in phase II clinical trials to terminate the trial early if the experimental treatment is futile. Interim monitoring requires that patients’ responses be ascertained shortly after the initiation of treatment so that the outcomes are known by the time the interim decision must be made. However, in some cases, response outcomes require a long time to be assessed, which causes difficulties for interim monitoring. To address this issue, we propose a Bayesian trial design to allow for continuously monitoring phase II clinical trials in the presence of delayed responses. We treat the delayed responses as missing data and handle them using a multiple imputation approach. Extensive simulations show that the proposed design yields desirable operating characteristics under various settings and dramatically reduces the trial duration. Copyright © 2014 John Wiley & Sons, Ltd.