Design and analysis of group-randomized trials: a review of recent methodological developments

We review recent developments in the design and analysis of group-randomized trials (GRTs). Regarding design, we summarize developments in estimates of intraclass correlation, power analysis, matched designs, designs involving one group per condition, and designs in which individuals are randomized to receive treatments in groups. Regarding analysis, we summarize developments in marginal and conditional models, the sandwich estimator, model-based estimators, binary data, survival analysis, randomization tests, survey methods, latent variable methods and nonlinear mixed models, time series methods, global tests for multiple endpoints, mediation effects, missing data, trial reporting, and software. We encourage investigators who conduct GRTs to become familiar with these developments and to collaborate with methodologists who can strengthen the design and analysis of their trials.     

Novel methods for the analysis of stepped wedge cluster randomized trials

Stepped wedge cluster randomized trials (SW-CRTs) have become increasingly popular and are used for a variety of interventions and outcomes, often chosen for their feasibility advantages. SW-CRTs must account for time trends in the outcome because of the staggered rollout of the intervention. Robust inference procedures and nonparametric analysis methods have recently been proposed to handle such trends without requiring strong parametric modeling assumptions, but these are less powerful than model-based approaches. We propose several novel analysis methods that reduce reliance on modeling assumptions while preserving some of the increased power provided by the use of mixed effects models. In one method, we use the synthetic control approach to find the best matching clusters for a given intervention cluster. Another method makes use of within-cluster crossover information to construct an overall estimator. We also consider methods that combine these approaches to further improve power. We test these methods on simulated SW-CRTs, describing scenarios in which these methods have increased power compared with existing nonparametric methods while preserving nominal validity when mixed effects models are misspecified. We also demonstrate theoretical properties of these estimators with less restrictive assumptions than mixed effects models. Finally, we propose avenues for future research on the use of these methods; motivation for such research arises from their flexibility, which allows the identification of specific causal contrasts of interest, their robustness, and the potential for incorporating covariates to further increase power. Investigators conducting SW-CRTs might well consider such methods when common modeling assumptions may not hold.     

An adaptive design for identifying the dose with the best efficacy/tolerability profile with application to a crossover dose‐finding study

Proof‐of‐concept in clinical trials has traditionally focused on the identification of a maximum tolerated dose with the assumption that the higher doses provide better efficacy. However, adverse events associated with a maximum tolerated dose may have a negative effect on efficacy. We present an efficient adaptive dose‐finding strategy that concentrates patient assignments at and around the dose which has the best efficacy/tolerability profile based on a utility function. The strategy is applied within the setting of a crossover design. While the strategy may also be applied to parallel studies, a crossover design provides more power for a given sample size for comparisons between the optimal dose versus placebo and/or active control when it is reasonable to assume no carryover effects. Copyright © 2009 John Wiley & Sons, Ltd.     

Model-based analyses of bioequivalence crossover trials using the stochastic approximation expectation maximisation algorithm

In this work, we develop a bioequivalence analysis using nonlinear mixed effects models (NLMEM) that mimics the standard noncompartmental analysis (NCA). We estimate NLMEM parameters, including between‐subject and within‐subject variability and treatment, period and sequence effects. We explain how to perform a Wald test on a secondary parameter, and we propose an extension of the likelihood ratio test for bioequivalence. We compare these NLMEM‐based bioequivalence tests with standard NCA‐based tests. We evaluate by simulation the NCA and NLMEM estimates and the type I error of the bioequivalence tests. For NLMEM, we use the stochastic approximation expectation maximisation (SAEM) algorithm implemented in monolix . We simulate crossover trials under H₀ using different numbers of subjects and of samples per subject. We simulate with different settings for between‐subject and within‐subject variability and for the residual error variance. The simulation study illustrates the accuracy of NLMEM‐based geometric means estimated with the SAEM algorithm, whereas the NCA estimates are biased for sparse design. NCA‐based bioequivalence tests show good type I error except for high variability. For a rich design, type I errors of NLMEM‐based bioequivalence tests (Wald test and likelihood ratio test) do not differ from the nominal level of 5%. Type I errors are inflated for sparse design. We apply the bioequivalence Wald test based on NCA and NLMEM estimates to a three‐way crossover trial, showing that Omnitrope®; (Sandoz GmbH, Kundl, Austria) powder and solution are bioequivalent to Genotropin®; (Pfizer Pharma GmbH, Karlsruhe, Germany). NLMEM‐based bioequivalence tests are an alternative to standard NCA‐based tests. However, caution is needed for small sample size and highly variable drug. Copyright © 2011 John Wiley & Sons, Ltd.     

An overview of variance inflation factors for sample-size calculation

For power and sample-size calculations, most practicing researchers rely on power and sample-size software programs to design their studies. There are many factors that affect the statistical power that, in many situations, go beyond the coverage of commercial software programs. Factors commonly known as design effects influence statistical power by inflating the variance of the test statistics. The authors quantify how these factors affect the variances so that researchers can adjust the statistical power or sample size accordingly. The authors review design effects for factorial design, crossover design, cluster randomization, unequal sample-size design, multiarm design, logistic regression, Cox regression, and the linear mixed model, as well as missing data in various designs. To design a study, researchers can apply these design effects, also known as variance inflation factors to adjust the power or sample size calculated from a two-group parallel design using standard formulas and software.     

Longitudinal structural mixed models for the analysis of surgical trials with noncompliance

Patient noncompliance complicates the analysis of many randomized trials seeking to evaluate the effect of surgical intervention as compared with a nonsurgical treatment. If selection for treatment depends on intermediate patient characteristics or outcomes, then 'as-treated' analyses may be biased for the estimation of causal effects. Therefore, the selection mechanism for treatment and/or compliance should be carefully considered when conducting analysis of surgical trials. We compare the performance of alternative methods when endogenous processes lead to patient crossover. We adopt an underlying longitudinal structural mixed model that is a natural example of a structural nested model. Likelihood-based methods are not typically used in this context; however, we show that standard linear mixed models will be valid under selection mechanisms that depend only on past covariate and outcome history. If there are underlying patient characteristics that influence selection, then likelihood methods can be extended via maximization of the joint likelihood of exposure and outcomes. Semi-parametric causal estimation methods such as marginal structural models, g-estimation, and instrumental variable approaches can also be valid, and we both review and evaluate their implementation in this setting. The assumptions required for valid estimation vary across approaches; thus, the choice of methods for analysis should be driven by which outcome and selection assumptions are plausible.     

Impact of modelling intra-subject variability on tests based on non-linear mixed-effects models in cross-over pharmacokinetic trials with application to the interaction of tenofovir on atazanavir in HIV patients

We evaluated the impact of modelling intra-subject variability on the likelihood ratio test (LRT) and the Wald test based on non-linear mixed effects models in pharmacokinetic interaction and bioequivalence cross-over trials. These tests were previously found to achieve a good power but an inflated type I error when intra-subject variability was not taken into account. Trials were simulated under H0 and several H1 and analysed with the NLME function. Different configurations of the number of subjects n and of the number of samples per subject J were evaluated for pharmacokinetic interaction and bioequivalence trials. Assuming intra-subject variability in the model dramatically improved the type I error of both interaction tests. For the Wald test, the type I error decreased from 22, 14 and 7.7 per cent for the original (n = 12, J = 10), intermediate (n = 24, J = 5) and sparse (n = 40, J = 3) designs, respectively, down to 7.5, 6.4 and 3.5 per cent when intra-subject variability was modelled. The LRT achieved very similar results. This improvement seemed mostly due to a better estimation of the standard error of the treatment effect. For J = 10, the type I error was found to be closer to 5 per cent when n increased when modelling intra-subject variability. Power was satisfactory for both tests. For bioequivalence trials, the type I error of the Wald test was 6.4, 5.7 and 4.2 per cent for the original, intermediate and sparse designs, respectively, when modelling intra-subject variability. We applied the Wald test to the pharmacokinetic interaction of tenofovir on atazanavir, a novel protease inhibitor. A significant decrease of the area under the curve of atazanavir was found when patients received tenofovir.     

An integrated population-averaged approach to the design,analysis and sample size determination of cluster-unit trials

While the mixed model approach to cluster randomization trials is relatively well developed, there has been less attention given to the design and analysis of population-averaged models for randomized and non-randomized cluster trials. We provide novel implementations of familiar methods to meet these needs. A design strategy that selects matching control communities based upon propensity scores, a statistical analysis plan for dichotomous outcomes based upon generalized estimating equations (GEE) with a design-based working correlation matrix, and new sample size formulae are applied to a large non-randomized study to reduce underage drinking. The statistical power calculations, based upon Wald tests for summary statistics, are special cases of a general power method for GEE.     

THE CASE FOR CROSS-OVER TRIALS IN PHASE III

There is a common belief that cross-over trials should not be used in phase III of drug development. This was reinforced by a statement in the draft CPMP Note for Guidance on biostatistical methodology in clinical trials which was circulated for review in March 1993: ‘Hence crossover trials in patients should be avoided as far as possible’. We do not share this belief. Historically, many successful drug developments in indications such as hypertension and asthma have depended heavily on cross-over trials in their phase III programmes, leading to regulatory approval for a number of well established medicines. The evidence on which these developments were based appeared sound at the time, and has not been questioned by later experiences with these medicines. Furthermore, the general level of understanding of these medical indications is now even more well developed, and hence the circumstances under which cross-over trials may be used to advantage for new drugs in phase III are even more likely to be correctly identified. There are some well-known disadvantages of cross-over trials relative to parallel group trials. These are reviewed and the ways in which early indications of such problems might be detected in phases I and II or elsewhere will be discussed. However, there are also two key advantages, the well-known one of study size and a less well-known one arising in the context of treatment-by-patient interaction. In phases I and II these advantages lead routinely to the use of the cross-over design. Some methods of analysing cross-over trials have been criticized in a number of recent articles. We compare the properties of a number of alternative analysis strategies by means of simulation and conclude that these concerns about methods of analysis do not imply that cross-over trials should be avoided, especially if baseline measurements can be included in the design. Any small risks attached to their use should not normally concern the regulator as they will tend to diminish estimates of treatment effects rather than enhance them. In summary, cross-over trials remain a potentially valuable research tool in the development of new medicines at all stages including phase III. It is unnecessary and counterproductive to exclude them from use.     

Evidence from crossover trials: empirical evaluation and comparison against parallel arm trials

BACKGROUND: We aimed to evaluate empirically how crossover trial results are analysed in meta-analyses of randomized evidence and whether their results agree with parallel arm studies on the same questions. METHODS: We used a systematic sample of Cochrane meta-analyses including crossover trials. We evaluated the methods of analysis for crossover results and compared the concordance of the estimated effect sizes in crossover vs parallel arm trials. RESULTS: Of 334 screened reviews, 62 had crossover trials. Of those, 33 meta-analyses performed quantitative syntheses involving two-arm two-period crossover trials. There was large variability on how these trials were analysed; only one of the 33 meta-analyses stated that they used the data from both the first and second period with an appropriate paired approach. Nine meta-analyses used the first period data only and 14 gave no information at all on what they had done. Twenty-eight meta-analyses had both crossover (n = 137, sample size n = 7,162) and parallel arm (n = 132, sample size n = 11,398) trials. Effect sizes correlated well with the two types of designs (rho = 0.72). Differences on whether the summary effect had a P < 0.05 or not were common due to limited sample sizes. The summary relative odds ratio for parallel arm vs crossover designs for favourable outcomes was 0.87 (95% CI, 0.74-1.02). CONCLUSIONS: Crossover designs may contribute evidence in a fifth of systematic reviews, but few meta-analyses make use of their full data. The results of crossover trials tend to agree with those of parallel arm trials, although there was a trend for more conservative treatment effect estimates in parallel arm trials.     

How many times should a cluster randomized crossover trial cross over?

Trial planning requires making efficient yet practical design choices. In a cluster randomized crossover trial, clusters of subjects cross back and forth between implementing the control and intervention conditions over the course of the trial, with each crossover marking the start of a new period. If it is possible to set up such a trial with more crossovers, a pertinent question is whether there are efficiency gains from clusters crossing over more frequently, and if these gains are substantial enough to justify the added complexity and cost of implementing more crossovers. We seek to determine the optimal number of crossovers for a fixed trial duration, and then identify other highly efficient designs by allowing the total number of clusters to vary and imposing thresholds on maximum cost and minimum statistical power. Our results pertain to trials with continuous recruitment and a continuous primary outcome, with the treatment effect estimated using a linear mixed model. To account for the similarity between subjects' outcomes within a cluster, we assume a correlation structure in which the correlation decays gradually in a continuous manner as the time between subjects' measurements increases. The optimal design is characterized by crossovers between the control and intervention conditions with each successive subject. However, this design is neither practical nor cost-efficient to implement, nor is it necessary: the gains in efficiency increase sharply in moving from a two-period to a four-period trial design, but approach an asymptote for the scenarios considered as the number of crossovers continues to increase.     

A sample size computation method for non-linear mixed effects models with applications to pharmacokinetics models

We propose a simple method to compute sample size for an arbitrary test hypothesis in population pharmacokinetics (PK) studies analysed with non-linear mixed effects models. Sample size procedures exist for linear mixed effects model, and have been recently extended by Rochon using the generalized estimating equation of Liang and Zeger. Thus, full model based inference in sample size computation has been possible. The method we propose extends the approach using a first-order linearization of the non-linear mixed effects model and use of the Wald chi(2) test statistic. The proposed method is general. It allows an arbitrary non-linear model as well as arbitrary distribution of random effects characterizing both inter- and intra-individual variability of the mixed effects model. To illustrate possible uses of the method we present tables of minimum sample sizes, in particular, with an illustration of the effect of sampling design on sample size. We demonstrate how (D-)optimal or frequent sampling requires fewer subjects in comparison to a sparse sampling design. We also present results from Monte Carlo simulations showing that the computed sample size can produce the desired power. The proposed method greatly reduces computing times compared with simulation-based methods of estimating sample sizes for population PK studies.     

Crossover designs for clinical trials

I discuss three-period crossover designs for an efficient comparison of two test treatments with special application to clinical trials which often have many practical limitations. In this paper I specify a subset of three-period crossover designs so that the investigators are not left with the problematic two-period two-sequence design, should the trials be terminated after the second period. I show that there is a dramatic reduction in variability for estimating the direct and residual treatment effects in three-period designs compared to two-period designs. I also show that the universally optimal design with ABB and BAA sequences is unsuitable when a complex form of residual effects is suspected, such as the second-order residual effects or treatment by period interactions. The design with ABB, BAA, AAB, and BBA sequences is relatively robust to these uncertain model assumptions. I also discuss missing data problems and conclude that, even with a large proportion of missing values, the three-period design is far more efficient than the two-period design.     

Stochastic variation in network epidemic models: implications for the design of community level HIV prevention trials

Important sources of variation in the spread of HIV in communities arise from overlapping sexual networks and heterogeneity in biological and behavioral risk factors in populations. These sources of variation are not routinely accounted for in the design of HIV prevention trials. In this paper, we use agent‐based models to account for these sources of variation. We illustrate the approach with an agent‐based model for the spread of HIV infection among men who have sex with men in South Africa. We find that traditional sample size approaches that rely on binomial (or Poisson) models are inadequate and can lead to underpowered studies. We develop sample size and power formulas for community randomized trials that incorporate estimates of variation determined from agent‐based models. We conclude that agent‐based models offer a useful tool in the design of HIV prevention trials. Copyright © 2014 John Wiley & Sons, Ltd.     

Sample size requirements for detecting treatment effect heterogeneity in cluster randomized trials

Cluster randomized trials (CRTs) refer to experiments with randomization carried out at the cluster or the group level. While numerous statistical methods have been developed for the design and analysis of CRTs, most of the existing methods focused on testing the overall treatment effect across the population characteristics, with few discussions on the differential treatment effect among subpopulations. In addition, the sample size and power requirements for detecting differential treatment effect in CRTs remain unclear, but are helpful for studies planned with such an objective. In this article, we develop a new sample size formula for detecting treatment effect heterogeneity in two-level CRTs for continuous outcomes, continuous or binary covariates measured at cluster or individual level. We also investigate the roles of two intraclass correlation coefficients (ICCs): the adjusted ICC for the outcome of interest and the marginal ICC for the covariate of interest. We further derive a closed-form design effect formula to facilitate the application of the proposed method, and provide extensions to accommodate multiple covariates. Extensive simulations are carried out to validate the proposed formula in finite samples. We find that the empirical power agrees well with the prediction across a range of parameter constellations, when data are analyzed by a linear mixed effects model with a treatment-by-covariate interaction. Finally, we use data from the HF-ACTION study to illustrate the proposed sample size procedure for detecting heterogeneous treatment effects.     

Design in nonlinear mixed effects models: optimization using the Fedorov-Wynn algorithm and power of the Wald test for binary covariates

We extend the methodology for designs evaluation and optimization in nonlinear mixed effects models with an illustration of the decrease of human immunodeficiency virus viral load after antiretroviral treatment initiation described by a bi-exponential model. We first show the relevance of the predicted standard errors (SEs) given by the computation of the population Fisher information matrix using the R function PFIM, in comparison to those computed with the stochastic approximation expectation-maximization algorithm, implemented in the Monolix software. We then highlight the usefulness of the Fedorov-Wynn (FW) algorithm for designs optimization compared to the Simplex algorithm. From the predicted SE of PFIM, we compute the predicted power of the Wald test to detect a treatment effect as well as the number of subjects needed to achieve a given power. Using the FW algorithm, we investigate the influence of the design on the power and show that, for optimized designs with the same total number of samples, the power increases when the number of subjects increases and the number of samples per subject decreases. A simulation study is also performed with the nlme function of R to confirm this result and show the relevance of the predicted powers compared to those observed by simulation.     

Robust analysis of stepped wedge trials using cluster‐level summaries within periods

In stepped‐wedge trials (SWTs), the intervention is rolled out in a random order over more than 1 time‐period. SWTs are often analysed using mixed‐effects models that require strong assumptions and may be inappropriate when the number of clusters is small. We propose a non‐parametric within‐period method to analyse SWTs. This method estimates the intervention effect by comparing intervention and control conditions in a given period using cluster‐level data corresponding to exposure. The within‐period intervention effects are combined with an inverse‐variance‐weighted average, and permutation tests are used. We present an example and, using simulated data, compared the method to (1) a parametric cluster‐level within‐period method, (2) the most commonly used mixed‐effects model, and (3) a more flexible mixed‐effects model. We simulated scenarios where period effects were common to all clusters, and when they varied according to a distribution informed by routinely collected health data. The non‐parametric within‐period method provided unbiased intervention effect estimates with correct confidence‐interval coverage for all scenarios. The parametric within‐period method produced confidence intervals with low coverage for most scenarios. The mixed‐effects models' confidence intervals had low coverage when period effects varied between clusters but had greater power than the non‐parametric within‐period method when period effects were common to all clusters. The non‐parametric within‐period method is a robust method for analysing SWT. The method could be used by trial statisticians who want to emphasise that the SWT is a randomised trial, in the common position of being uncertain about whether data will meet the assumptions necessary for mixed‐effect models.     

Analysis of case-crossover designs using longitudinal approaches: a simulation study

BACKGROUND: Application of case-crossover designs provides an alternative to time-series analysis for analyzing the health-related effects of air pollution. Although some case-crossover studies can control for trend and seasonality by design, to date they have been analyzed as matched case-control studies. Such analyses may exhibit biases and a lower statistical efficiency than traditional time series analyzed with Poisson. METHODS: In this article, case-crossover studies are treated as cohort studies in which each subject is observed for a short period of time before and/or after the event, thus making possible analyzing with Andersen-Gill and generalized linear mixed models. We conducted a simulation study to compare the behavior of these models applied to case-crossover designs with time series analyzed with Poisson and with case-crossover analyzed by conditional logistic regression. To this end, we created a random variable that follows a Poisson distribution of low (2/day) and high mean events (22/day). This variable is a function of an unobserved confounding variable (that introduces trend and seasonality) and data on small particulate matter (PM10) from Barcelona. In addition, scenarios were created to assess the effect on exposure exerted by autocorrelation and the magnitude of the pollutant coefficient. RESULTS: The full semisymmetric design analyzed with generalized linear mixed models yields good coverage and a high statistical power for air-pollution effect magnitudes close to the real values but shows bias for high effect magnitudes. This bias seems to be attributable to autocorrelation in the exposure variable. CONCLUSIONS: Longitudinal approaches applied to case-crossover designs may prove useful for analyzing the acute effects of environmental exposures.     

The power prior: theory and applications

The power prior has been widely used in many applications covering a large number of disciplines. The power prior is intended to be an informative prior constructed from historical data. It has been used in clinical trials, genetics, health care, psychology, environmental health, engineering, economics, and business. It has also been applied for a wide variety of models and settings, both in the experimental design and analysis contexts. In this review article, we give an A‐to‐Z exposition of the power prior and its applications to date. We review its theoretical properties, variations in its formulation, statistical contexts for which it has been used, applications, and its advantages over other informative priors. We review models for which it has been used, including generalized linear models, survival models, and random effects models. Statistical areas where the power prior has been used include model selection, experimental design, hierarchical modeling, and conjugate priors. Frequentist properties of power priors in posterior inference are established, and a simulation study is conducted to further examine the empirical performance of the posterior estimates with power priors. Real data analyses are given illustrating the power prior as well as the use of the power prior in the Bayesian design of clinical trials. Copyright © 2015 John Wiley & Sons, Ltd.