A covariate‐adjustment regression model approach to noninferiority margin definition

To maintain the interpretability of the effect of experimental treatment (EXP) obtained from a noninferiority trial, current statistical approaches often require the constancy assumption. This assumption typically requires that the control treatment effect in the population of the active control trial is the same as its effect presented in the population of the historical trial. To prevent constancy assumption violation, clinical trial sponsors were recommended to make sure that the design of the active control trial is as close to the design of the historical trial as possible. However, these rigorous requirements are rarely fulfilled in practice. The inevitable discrepancies between the historical trial and the active control trial have led to debates on many controversial issues. Without support from a well‐developed quantitative method to determine the impact of the discrepancies on the constancy assumption violation, a correct judgment seems difficult. In this paper, we present a covariate‐adjustment generalized linear regression model approach to achieve two goals: (1) to quantify the impact of population difference between the historical trial and the active control trial on the degree of constancy assumption violation and (2) to redefine the active control treatment effect in the active control trial population if the quantification suggests an unacceptable violation. Through achieving goal (1), we examine whether or not a population difference leads to an unacceptable violation. Through achieving goal (2), we redefine the noninferiority margin if the violation is unacceptable. This approach allows us to correctly determine the effect of EXP in the noninferiority trial population when constancy assumption is violated due to the population difference. We illustrate the covariate‐adjustment approach through a case study. Copyright © 2010 John Wiley & Sons, Ltd.     

Sample size calculation for the one‐sample log‐rank test

An improved method of sample size calculation for the one‐sample log‐rank test is provided. The one‐sample log‐rank test may be the method of choice if the survival curve of a single treatment group is to be compared with that of a historic control. Such settings arise, for example, in clinical phase‐II trials if the response to a new treatment is measured by a survival endpoint. Present sample size formulas for the one‐sample log‐rank test are based on the number of events to be observed, that is, in order to achieve approximately a desired power for allocated significance level and effect the trial is stopped as soon as a certain critical number of events are reached. We propose a new stopping criterion to be followed. Both approaches are shown to be asymptotically equivalent. For small sample size, though, a simulation study indicates that the new criterion might be preferred when planning a corresponding trial. In our simulations, the trial is usually underpowered, and the aspired significance level is not exploited if the traditional stopping criterion based on the number of events is used, whereas a trial based on the new stopping criterion maintains power with the type‐I error rate still controlled. Copyright © 2014 John Wiley & Sons, Ltd.     

Strength of evidence of non‐inferiority trials—The adjustment of the type I error rate in non‐inferiority trials with the synthesis method

In non‐inferiority trials that employ the synthesis method several types of dependencies among test statistics occur due to sharing of the same information from the historical trial. The conditions under which the dependencies appear may be divided into three categories. The first case is when a new drug is approved with single non‐inferiority trial. The second case is when a new drug is approved if two independent non‐inferiority trials show positive results. The third case is when two new different drugs are approved with the same active control. The problem of the dependencies is that they can make the type I error rate deviate from the nominal level. In order to study such deviations, we introduce the unconditional and conditional across‐trial type I error rates when the non‐inferiority margin is estimated from the historical trial, and investigate how the dependencies affect the type I error rates. We show that the unconditional across‐trial type I error rate increases dramatically as does the correlation between two non‐inferiority tests when a new drug is approved based on the positive results of two non‐inferiority trials. We conclude that the conditional across‐trial type I error rate involves the unknown treatment effect in the historical trial. The formulae of the conditional across‐trial type I error rates provide us with a way of investigating the conditional across‐trial type I error rates for various assumed values of the treatment effect in the historical trial. Copyright © 2010 John Wiley & Sons, Ltd.     

TACT method for non-inferiority testing in active controlled trials

In active controlled trials without a placebo arm, non-inferiority testing is often considered but has different objectives. For the objective of demonstrating the efficacy of an experimental treatment or retention of a fraction of the control effect by the treatment, there are two types of statistical methods for testing - the synthesis method and the confidence interval method. According to the study of Wang, Hung and Tsong, the former is efficient under the so-called constancy condition but may have the alpha error rate inflate rapidly if the condition does not hold. In contrast, the latter method with careful selection of the non-inferiority margin tends to be conservative if the condition holds and may still have a valid alpha error otherwise unless the effect of the active control is less to a large extent in the active controlled trial than in the historical trials. We developed the TACT method, Two-stage Active Control Testing, as a viable compromise between the two methods. Through the TACT method, the uninterpretable non-inferiority testing may be avoided prior to the end of the trial. The TACT method carefully constructed can have a valid alpha error rate and the power close to the synthesis method if the constancy condition holds. In addition, the TACT method is more powerful than the confidence interval method for testing for the efficacy of the new treatment relative to the putative placebo and for showing that the new treatment is not inferior to the active control comparator.     

One‐stage individual participant data meta‐analysis models: estimation of treatment‐covariate interactions must avoid ecological bias by separating out within‐trial and across‐trial information

Stratified medicine utilizes individual‐level covariates that are associated with a differential treatment effect, also known as treatment‐covariate interactions. When multiple trials are available, meta‐analysis is used to help detect true treatment‐covariate interactions by combining their data. Meta‐regression of trial‐level information is prone to low power and ecological bias, and therefore, individual participant data (IPD) meta‐analyses are preferable to examine interactions utilizing individual‐level information. However, one‐stage IPD models are often wrongly specified, such that interactions are based on amalgamating within‐ and across‐trial information. We compare, through simulations and an applied example, fixed‐effect and random‐effects models for a one‐stage IPD meta‐analysis of time‐to‐event data where the goal is to estimate a treatment‐covariate interaction. We show that it is crucial to centre patient‐level covariates by their mean value in each trial, in order to separate out within‐trial and across‐trial information. Otherwise, bias and coverage of interaction estimates may be adversely affected, leading to potentially erroneous conclusions driven by ecological bias. We revisit an IPD meta‐analysis of five epilepsy trials and examine age as a treatment effect modifier. The interaction is ?0.011 (95% CI: ?0.019 to ?0.003; p = 0.004), and thus highly significant, when amalgamating within‐trial and across‐trial information. However, when separating within‐trial from across‐trial information, the interaction is ?0.007 (95% CI: ?0.019 to 0.005; p = 0.22), and thus its magnitude and statistical significance are greatly reduced. We recommend that meta‐analysts should only use within‐trial information to examine individual predictors of treatment effect and that one‐stage IPD models should separate within‐trial from across‐trial information to avoid ecological bias. © 2016 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.     

A weighted combined effect measure for the analysis of a composite time‐to‐first‐event endpoint with components of different clinical relevance

Composite endpoints combine several events within a single variable, which increases the number of expected events and is thereby meant to increase the power. However, the interpretation of results can be difficult as the observed effect for the composite does not necessarily reflect the effects for the components, which may be of different magnitude or even point in adverse directions. Moreover, in clinical applications, the event types are often of different clinical relevance, which also complicates the interpretation of the composite effect. The common effect measure for composite endpoints is the all‐cause hazard ratio, which gives equal weight to all events irrespective of their type and clinical relevance. Thereby, the all‐cause hazard within each group is given by the sum of the cause‐specific hazards corresponding to the individual components. A natural extension of the standard all‐cause hazard ratio can be defined by a “weighted all‐cause hazard ratio” where the individual hazards for each component are multiplied with predefined relevance weighting factors. For the special case of equal weights across the components, the weighted all‐cause hazard ratio then corresponds to the standard all‐cause hazard ratio. To identify the cause‐specific hazard of the individual components, any parametric survival model might be applied. The new weighted effect measure can be tested for deviations from the null hypothesis by means of a permutation test. In this work, we systematically compare the new weighted approach to the standard all‐cause hazard ratio by theoretical considerations, Monte‐Carlo simulations, and by means of a real clinical trial example.     

Estimating long‐term effects of treatment from placebo‐controlled trials with an extension period,using virtual twins

The best information about the benefits of long‐term treatment is obtained from a long‐term placebo‐controlled trial. However, once efficacy has been demonstrated in relatively brief trials, it may not be possible to conduct long‐term placebo‐controlled trials, for ethical or practical reasons. This paper presents a method for estimating long‐term effects of a treatment from a placebo‐controlled trial in which some participants originally randomized to active‐treatment volunteer to continue on treatment during an extension study, but follow‐up of participants originally assigned to placebo ends with the trial, or they are crossed over to active treatment during the extension. We propose using data from the trial to project the outcomes for a ‘virtual twin’ for each active‐treatment volunteer under the counterfactual placebo condition, and using bootstrap methods for inference. The proposed method is validated using simulation, and applied to data from the Fracture Intervention Trial and its extension, FLEX. Copyright © 2010 John Wiley & Sons, Ltd.     

Multiplicative interaction in network meta‐analysis

Meta‐analysis of a set of clinical trials is usually conducted using a linear predictor with additive effects representing treatments and trials. Additivity is a strong assumption. In this paper, we consider models for two or more treatments that involve multiplicative terms for interaction between treatment and trial. Multiplicative models provide information on the sensitivity of each treatment effect relative to the trial effect. In developing these models, we make use of a two‐way analysis‐of‐variance approach to meta‐analysis and consider fixed or random trial effects. It is shown using two examples that models with multiplicative terms may fit better than purely additive models and provide insight into the nature of the trial effect. We also show how to model inconsistency using multiplicative terms. Copyright © 2014 John Wiley & Sons, Ltd.     

Power and sample size calculation for log‐rank test with a time lag in treatment effect

The log‐rank test is the most powerful non‐parametric test for detecting a proportional hazards alternative and thus is the most commonly used testing procedure for comparing time‐to‐event distributions between different treatments in clinical trials. When the log‐rank test is used for the primary data analysis, the sample size calculation should also be based on the test to ensure the desired power for the study. In some clinical trials, the treatment effect may not manifest itself right after patients receive the treatment. Therefore, the proportional hazards assumption may not hold. Furthermore, patients may discontinue the study treatment prematurely and thus may have diluted treatment effect after treatment discontinuation. If a patient's treatment termination time is independent of his/her time‐to‐event of interest, the termination time can be treated as a censoring time in the final data analysis. Alternatively, we may keep collecting time‐to‐event data until study termination from those patients who discontinued the treatment and conduct an intent‐to‐treat analysis by including them in the original treatment groups. We derive formulas necessary to calculate the asymptotic power of the log‐rank test under this non‐proportional hazards alternative for the two data analysis strategies. Simulation studies indicate that the formulas provide accurate power for a variety of trial settings. A clinical trial example is used to illustrate the application of the proposed methods. Copyright © 2009 John Wiley & Sons, Ltd.     

Reference‐based multiple imputation for missing data sensitivity analyses in trial‐based cost‐effectiveness analysis

Missing data are a common issue in cost‐effectiveness analysis (CEA) alongside randomised trials and are often addressed assuming the data are ‘missing at random’. However, this assumption is often questionable, and sensitivity analyses are required to assess the implications of departures from missing at random. Reference‐based multiple imputation provides an attractive approach for conducting such sensitivity analyses, because missing data assumptions are framed in an intuitive way by making reference to other trial arms. For example, a plausible not at random mechanism in a placebo‐controlled trial would be to assume that participants in the experimental arm who dropped out stop taking their treatment and have similar outcomes to those in the placebo arm. Drawing on the increasing use of this approach in other areas, this paper aims to extend and illustrate the reference‐based multiple imputation approach in CEA. It introduces the principles of reference‐based imputation and proposes an extension to the CEA context. The method is illustrated in the CEA of the CoBalT trial evaluating cognitive behavioural therapy for treatment‐resistant depression. Stata code is provided. We find that reference‐based multiple imputation provides a relevant and accessible framework for assessing the robustness of CEA conclusions to different missing data assumptions.     

A Bayesian model averaging approach with non‐informative priors for cost‐effectiveness analyses

We consider the problem of assessing new and existing technologies for their cost‐effectiveness in the case where data on both costs and effects are available from a clinical trial, and we address it by means of the cost‐effectiveness acceptability curve. The main difficulty in these analyses is that cost data usually exhibit highly skew and heavy‐tailed distributions, so that it can be extremely difficult to produce realistic probabilistic models for the underlying population distribution. Here, in order to integrate the uncertainty about the model into the analysis of cost data and into cost‐effectiveness analyses, we consider an approach based on Bayesian model averaging (BMA) in the particular case of weak prior informations about the unknown parameters of the different models involved in the procedure. The main consequence of this assumption is that the marginal densities required by BMA are undetermined. However, in accordance with the theory of partial Bayes factors and in particular of fractional Bayes factors, we suggest replacing each marginal density with a ratio of integrals that can be efficiently computed via path sampling. Copyright © 2010 John Wiley & Sons, Ltd.     

Sample size calculations for micro‐randomized trials in mHealth

The use and development of mobile interventions are experiencing rapid growth. In “just‐in‐time” mobile interventions, treatments are provided via a mobile device, and they are intended to help an individual make healthy decisions ‘in the moment,’ and thus have a proximal, near future impact. Currently, the development of mobile interventions is proceeding at a much faster pace than that of associated data science methods. A first step toward developing data‐based methods is to provide an experimental design for testing the proximal effects of these just‐in‐time treatments. In this paper, we propose a ‘micro‐randomized’ trial design for this purpose. In a micro‐randomized trial, treatments are sequentially randomized throughout the conduct of the study, with the result that each participant may be randomized at the 100s or 1000s of occasions at which a treatment might be provided. Further, we develop a test statistic for assessing the proximal effect of a treatment as well as an associated sample size calculator. We conduct simulation evaluations of the sample size calculator in various settings. Rules of thumb that might be used in designing a micro‐randomized trial are discussed. This work is motivated by our collaboration on the HeartSteps mobile application designed to increase physical activity. Copyright © 2015 John Wiley & Sons, Ltd.     

A study to establish whether food‐based approaches can improve serum iron levels in child‐bearing aged women

Background: Iron‐deficiency anaemia is particularly prevalent in pregnancy. The present study aimed to determine whether functional bread containing teff flour (i.e. naturally rich in iron) could be an alternative way of improving iron status. However, before testing whether its consumption can improve pregnancy iron status, the bio‐availability of iron was determined in a sample of nonpregnant women. Methods: Fifty‐eight women (20–50 years) were recruited from the University. Blood samples were taken at baseline to assess iron status and participants were screened to account for other factors affecting iron status. Twenty‐four participants (haemoglobin 9.5–14.0 g dL^?1) were recruited to take part in the intervention and allocated to five groups: (i) control bread (CB); (ii) teff bread (TB); (iii) TB + level 1 phytase (TB + P1); (iv) TB + level 2 phytase (TB + P2); or (v) a supplement containing 10 mg of ferrous sulphate. Venous blood samples were taken before the intervention and after 180–210 min, aiming to determine changes in serum iron. Results: Consuming three or four slices of TB provided statistically significantly more iron (7.6 mg) than CB (5.1 mg) (P < 0.001). Because participants were fasted, serum iron levels declined in all bread groups (average ?1.5 μm ), although the smallest reduction was observed in the TB + P2 group (?0.3 μm ). The area‐under‐the‐curve from baseline to 210 min was lower in the TB + P2 (?78.8 μmol min L^?1) group compared to the other bread interventions, indicating higher levels of iron absorption in this group. Conclusions: The results obtained in the present study show that TB consumption may help to maintain serum iron levels, especially when phytase is added. The findings from the study also demonstrate there may be potential to further improve the bio‐availability of iron from non‐haem food sources.     

Meta‐analysis of time‐to‐event outcomes from randomized trials using restricted mean survival time: application to individual participant data

Meta‐analysis of time‐to‐event outcomes using the hazard ratio as a treatment effect measure has an underlying assumption that hazards are proportional. The between‐arm difference in the restricted mean survival time is a measure that avoids this assumption and allows the treatment effect to vary with time. We describe and evaluate meta‐analysis based on the restricted mean survival time for dealing with non‐proportional hazards and present a diagnostic method for the overall proportional hazards assumption. The methods are illustrated with the application to two individual participant meta‐analyses in cancer. The examples were chosen because they differ in disease severity and the patterns of follow‐up, in order to understand the potential impacts on the hazards and the overall effect estimates. We further investigate the estimation methods for restricted mean survival time by a simulation study. Copyright © 2015 John Wiley & Sons, Ltd.     

A framework for considering the risk‐benefit trade‐off in designing noninferiority trials using composite outcome approaches

When a new treatment regimen is expected to have comparable or slightly worse efficacy to that of the control regimen but has benefits in other domains such as safety and tolerability, a noninferiority (NI) trial may be appropriate but is fraught with difficulty in justifying an acceptable NI margin that is based on both clinical and statistical input. To overcome this, we propose to utilize composite risk‐benefit outcomes that combine elements from domains of importance (eg, efficacy, safety, and tolerability). The composite outcome itself may be analyzed using a superiority framework, or it can be used as a tool at the design stage of a NI trial for selecting an NI margin for efficacy that balances changes in risks and benefits. In the latter case, the choice of NI margin may be based on a novel quantity called the maximum allowable decrease in efficacy (MADE), defined as the marginal difference in efficacy between arms that would yield a null treatment effect for the composite outcome given an assumed distribution for the composite outcome. We observe that MADE: (1) is larger when the safety improvement for the experimental arm is larger, (2) depends on the association between the efficacy and safety outcomes, and (3) depends on the control arm efficacy rate. We use a numerical example for power comparisons between a superiority test for the composite outcome vs a noninferiority test for efficacy using the MADE as the NI margin, and apply the methods to a TB treatment trial.     

Health benefits for health and social care clients attending an Integrated Health and Social Care day unit (IHSCDU): a before‐and‐after pilot study with a comparator group

It is thought that integrating health and social care provision can improve services, yet few evaluations of integrated health and social care initiatives have focused on changes in clinical outcomes and used comparator groups. The aim of this pilot study was to identify whether attendance at an integrated health and social care day unit (IHSCDU) affected selected outcomes of functional mobility, number of prescribed medications, and physical and psychological well‐being. A secondary aim was to examine the utility of the tools to measure these outcomes in this context; the feasibility of the recruitment and retention strategy and the utility of the comparator group. A before‐and‐after comparison design was used with non‐randomised intervention and comparator arms. The intervention arm comprised 30 service users attending the IHSCDU and the comparator arm comprised 33 service users on a community nursing caseload. Measures of functional mobility (Barthel's Index) and physical and psychological well‐being (SF‐12^®) were taken from all participants in both arms at three data collection points: baseline, 4 and 9 months later, between November 2010 and September 2012. Participants and outcomes were identified prospectively and in both arms, the individual was the unit of assignment. No significant changes were noted in functional mobility and psychological well‐being and the number of medications prescribed increased in both arms. There was a trend towards a significant difference between study arms in the change in the SF‐12^® physical health outcome measure and this outcome measure could be usefully explored in future studies. The recruitment and retention strategy was feasible although our comparator group had some limitations in not being closely matched in terms of age, functional mobility and mental well‐being.     

Using structural‐nested models to estimate the effect of cluster‐level adherence on individual‐level outcomes with a three‐armed cluster‐randomized trial

Much attention has been paid to estimating the causal effect of adherence to a randomized protocol using instrumental variables to adjust for unmeasured confounding. Researchers tend to use the instrumental variable within one of the three main frameworks: regression with an endogenous variable, principal stratification, or structural‐nested modeling. We found in our literature review that even in simple settings, causal interpretations of analyses with endogenous regressors can be ambiguous or rely on a strong assumption that can be difficult to interpret. Principal stratification and structural‐nested modeling are alternative frameworks that render unambiguous causal interpretations based on assumptions that are, arguably, easier to interpret. Our interest stems from a wish to estimate the effect of cluster‐level adherence on individual‐level binary outcomes with a three‐armed cluster‐randomized trial and polytomous adherence. Principal stratification approaches to this problem are quite challenging because of the sheer number of principal strata involved. Therefore, we developed a structural‐nested modeling approach and, in the process, extended the methodology to accommodate cluster‐randomized trials with unequal probability of selecting individuals. Furthermore, we developed a method to implement the approach with relatively simple programming. The approach works quite well, but when the structural‐nested model does not fit the data, there is no solution to the estimating equation. We investigate the performance of the approach using simulated data, and we also use the approach to estimate the effect on pupil absence of school‐level adherence to a randomized water, sanitation, and hygiene intervention in western Kenya. Copyright © 2013 John Wiley & Sons, Ltd.     

Sample size calculation for meta‐epidemiological studies

Meta‐epidemiological studies are used to compare treatment effect estimates between randomized clinical trials with and without a characteristic of interest. To our knowledge, there is presently nothing to help researchers to a priori specify the required number of meta‐analyses to be included in a meta‐epidemiological study. We derived a theoretical power function and sample size formula in the framework of a hierarchical model that allows for variation in the impact of the characteristic between trials within a meta‐analysis and between meta‐analyses. A simulation study revealed that the theoretical function overestimated power (because of the assumption of equal weights for each trial within and between meta‐analyses). We also propose a simulation approach that allows for relaxing the constraints used in the theoretical approach and is more accurate. We illustrate that the two variables that mostly influence power are the number of trials per meta‐analysis and the proportion of trials with the characteristic of interest. We derived a closed‐form power function and sample size formula for estimating the impact of trial characteristics in meta‐epidemiological studies. Our analytical results can be used as a ‘rule of thumb’ for sample size calculation for a meta‐epidemiologic study. A more accurate sample size can be derived with a simulation study. Copyright © 2015 John Wiley & Sons, Ltd.     

An adaptive design for identifying the dose with the best efficacy/tolerability profile with application to a crossover dose‐finding study

Proof‐of‐concept in clinical trials has traditionally focused on the identification of a maximum tolerated dose with the assumption that the higher doses provide better efficacy. However, adverse events associated with a maximum tolerated dose may have a negative effect on efficacy. We present an efficient adaptive dose‐finding strategy that concentrates patient assignments at and around the dose which has the best efficacy/tolerability profile based on a utility function. The strategy is applied within the setting of a crossover design. While the strategy may also be applied to parallel studies, a crossover design provides more power for a given sample size for comparisons between the optimal dose versus placebo and/or active control when it is reasonable to assume no carryover effects. Copyright © 2009 John Wiley & Sons, Ltd.     

An evaluation of constrained randomization for the design and analysis of group‐randomized trials

In group‐randomized trials, a frequent practical limitation to adopting rigorous research designs is that only a small number of groups may be available, and therefore, simple randomization cannot be relied upon to balance key group‐level prognostic factors across the comparison arms. Constrained randomization is an allocation technique proposed for ensuring balance and can be used together with a permutation test for randomization‐based inference. However, several statistical issues have not been thoroughly studied when constrained randomization is considered. Therefore, we used simulations to evaluate key issues including the following: the impact of the choice of the candidate set size and the balance metric used to guide randomization; the choice of adjusted versus unadjusted analysis; and the use of model‐based versus randomization‐based tests. We conducted a simulation study to compare the type I error and power of the F‐test and the permutation test in the presence of group‐level potential confounders. Our results indicate that the adjusted F‐test and the permutation test perform similarly and slightly better for constrained randomization relative to simple randomization in terms of power, and the candidate set size does not substantially affect their power. Under constrained randomization, however, the unadjusted F‐test is conservative, while the unadjusted permutation test carries the desired type I error rate as long as the candidate set size is not too small; the unadjusted permutation test is consistently more powerful than the unadjusted F‐test and gains power as candidate set size changes. Finally, we caution against the inappropriate specification of permutation distribution under constrained randomization. An ongoing group‐randomized trial is used as an illustrative example for the constrained randomization design. Copyright © 2015 John Wiley & Sons, Ltd.