Sample size calculations for a split-cluster, beta-binomial design in the assessment of toxicity

Mouse embryo assays are recommended to test materials used for in vitro fertilization for toxicity. In such assays, a number of embryos is divided in a control group, which is exposed to a neutral medium, and a test group, which is exposed to a potentially toxic medium. Inferences on toxicity are based on observed differences in successful embryo development between the two groups. However, mouse embryo assays tend to lack power due to small group sizes. This paper focuses on the sample size calculations for one such assay, the Nijmegen mouse embryo assay (NMEA), in order to obtain an efficient and statistically validated design. The NMEA follows a stratified (mouse), randomized (embryo), balanced design (also known as a split-cluster design). We adopted a beta-binomial approach and obtained a closed sample size formula based on an estimator for the within-cluster variance. Our approach assumes that the average success rate of the mice and the variance thereof, which are breed characteristics that can be easily estimated from historical data, are known. To evaluate the performance of the sample size formula, a simulation study was undertaken which suggested that the predicted sample size was quite accurate. We confirmed that incorporating the a priori knowledge and exploiting the intra-cluster correlations enable a smaller sample size. Also, we explored some departures from the beta-binomial assumption. First, departures from the compound beta-binomial distribution to an arbitrary compound binomial distribution lead to the same formulas, as long as some general assumptions hold. Second, our sample size formula compares to the one derived from a linear mixed model for continuous outcomes in case the compound (beta-)binomial estimator is used for the within-cluster variance.     

Sample size calculation for comparing two negative binomial rates

Negative binomial model has been increasingly used to model the count data in recent clinical trials. It is frequently chosen over Poisson model in cases of overdispersed count data that are commonly seen in clinical trials. One of the challenges of applying negative binomial model in clinical trial design is the sample size estimation. In practice, simulation methods have been frequently used for sample size estimation. In this paper, an explicit formula is developed to calculate sample size based on the negative binomial model. Depending on different approaches to estimate the variance under null hypothesis, three variations of the sample size formula are proposed and discussed. Important characteristics of the formula include its accuracy and its ability to explicitly incorporate dispersion parameter and exposure time. The performance of the formula with each variation is assessed using simulations. Copyright © 2013 John Wiley & Sons, Ltd.     

Sample size calculations for clustered binary data.

In this paper we propose a sample size calculation method for testing on a binomial proportion when binary observations are dependent within clusters. In estimating the binomial proportion in clustered binary data, two weighting systems have been popular: equal weights to clusters and equal weights to units within clusters. When the number of units varies cluster by cluster, performance of these two weighting systems depends on the extent of correlation among units within each cluster. In addition to them, we will also use an optimal weighting method that minimizes the variance of the estimator. A sample size formula is derived for each of the estimators with different weighting schemes. We apply these methods to the sample size calculation for the sensitivity of a periodontal diagnostic test. Simulation studies are conducted to evaluate a finite sample performance of the three estimators. We also assess the influence of misspecified input parameter values on the calculated sample size. The optimal estimator requires equal or smaller sample sizes and is more robust to the misspecification of an input parameter than those assigning equal weights to units or clusters.     

The binomial distribution of meta-analysis was preferred to model within-study variability

OBJECTIVE: When studies report proportions such as sensitivity or specificity, it is customary to meta-analyze them using the DerSimonian and Laird random effects model. This method approximates the within-study variability of the proportion by a normal distribution, which may lead to bias for several reasons. Alternatively an exact likelihood approach based on the binomial within-study distribution can be used. This method can easily be performed in standard statistical packages. We investigate the performance of the standard method and the alternative approach. STUDY DESIGN AND SETTING: We compare the two approaches through a simulation study, in terms of bias, mean-squared error, and coverage probabilities. We varied the size of the overall sensitivity or specificity, the between-studies variance, the within-study sample sizes, and the number of studies. The methods are illustrated using a published meta-analysis data set. RESULTS: The exact likelihood approach performs always better than the approximate approach and gives unbiased estimates. The coverage probability, in particular for the profile likelihood, is also reasonably acceptable. In contrast, the approximate approach gives huge bias with very poor coverage probability in many cases. CONCLUSION: The exact likelihood approach is the method of preference and should be used whenever feasible.     

Allocation of subjects to test null relative risks smaller than one

Allocating a proportion k'=1/(1+ radicalr(0)) of subjects to an intervention is a practical approach to approximately maximize power for testing whether an intervention reduces relative risk of disease below a null ratio r(0)<1. Furthermore, allocating k'(s), a convenient fraction close to k', to intervention performs nearly as well; for example, allocating k'(s)=3/5 for 0.5> or =r(0)>0.33,2/3 for 0.33> or =r(0)>0.17 and 3/4 for 0.17> or =r(0)> or =0.10. Both k' and k'(s) are easily calculated and invariant to alterations in disease rate estimates under null and alternative hypotheses, when r(0) remains constant. In examples that we studied, allocating k' (or k'(s)) subjects to intervention achieved close to the minimum possible sample size, given test size and power (equivalently, maximum power, given test size and sample size), for likelihood score tests. Compared to equal allocation, k' and k'(s) reduced sample sizes by amounts ranging from approximately 5.5 per cent for r(0)=0.50 to approximately 24 per cent for r(0)=0.10. These sample size savings may be particularly important for large studies of prophylactic interventions such as vaccines. While k' was derived from variance minimization for an arcsine transformation, we do not recommend the arcsine test, since its true size exceeded the nominal value. In contrast, the true size for the uncorrected score test was less than the nominal size. A skewness correction made the size of the score test very close to the nominal level and slightly increased power. We recommend using the score test, or the skewness-corrected score test, for planing studies designed to show a ratio of proportions is less than a prespecified null ratio r(0)<1.     

Exact tests of equivalence and efficacy with a non-zero lower bound for comparative studies

Exact tests of equivalence and efficacy with a non-zero lower bound based on two independent binomial proportions for comparative trials are proposed. These exact tests are desirable for studies with small sample sizes. They generalize classical methods to include testing of null hypotheses of prespecified differences and can be used to demonstrate a new treatment's efficacy or its equivalence to a standard treatment. The proposed exact tests use unconditional distributions of the test statistics. Variances of test statistics are estimated via a constrained maximum likelihood method (Farrington and Manning). Data from oncology and vaccine clinical trials are used to illustrate the exact tests. © 1998 John Wiley & Sons, Ltd.     

On estimation of the variance in Cochran-Armitage trend tests for genetic association using case-control studies

The Cochran-Armitage trend test has been used in case-control studies for testing genetic association. As the variance of the test statistic is a function of unknown parameters, e.g. disease prevalence and allele frequency, it must be estimated. The usual estimator combining data for cases and controls assumes they follow the same distribution under the null hypothesis. Under the alternative hypothesis, however, the cases and controls follow different distributions. Thus, the power of the trend tests may be affected by the variance estimator used. In particular, the usual method combining both cases and controls is not an asymptotically unbiased estimator of the null variance when the alternative is true. Two different estimates of the null variance are available which are consistent under both the null and alternative hypotheses. In this paper, we examine sample size and small sample power performance of trend tests, which are optimal for three common genetic models as well as a robust trend test based on the three estimates of the variance and provide guidelines for choosing an appropriate test.     

A simple hybrid variance estimator for the Kaplan-Meier survival function

In this paper, we propose a hybrid variance estimator for the Kaplan-Meier survival function. This new estimator approximates the true variance by a Binomial variance formula, where the proportion parameter is a piecewise non-increasing function of the Kaplan-Meier survival function and its upper bound, as described below. Also, the effective sample size equals the number of subjects not censored prior to that time. In addition, we consider an adjusted hybrid variance estimator that modifies the regular estimator for small sample sizes. We present a simulation study to compare the performance of the regular and adjusted hybrid variance estimators to the Greenwood and Peto variance estimators for small sample sizes. We show that on average these hybrid variance estimators give closer variance estimates to the true values than the traditional variance estimators, and hence confidence intervals constructed with these hybrid variance estimators have more nominal coverage rates. Indeed, the Greenwood and Peto variance estimators can substantially underestimate the true variance in the left and right tails of the survival distribution, even with moderately censored data. Finally, we illustrate the use of these hybrid and traditional variance estimators on a data set from a leukaemia clinical trial.     

Estimation of the average correlation coefficient for stratified bivariate data

If the relationship between two ordered categorical variables X and Y is influenced by a third categorical variable with K levels, the Cochran-Mantel-Haenszel (CMH) correlation statistic QC is a useful stratum-adjusted summary statistic for testing the null hypothesis of no association between X and Y. Although motivated by and developed for the case of K I x J contingency tables, the correlation statistic QC is also applicable when X and Y are continuous variables. In this paper we derive a corresponding estimator of the average correlation coefficient for K I x J tables. We also study two estimates of the variance of the average correlation coefficient. The first is a restricted variance based on the variances of the observed cell frequencies under the null hypothesis of no association. The second is an unrestricted variance based on an asymptotic variance derived by Brown and Benedetti. The estimator of the average correlation coefficient works well in tables with balanced and unbalanced margins, for equal and unequal stratum-specific sample sizes, when correlation coefficients are constant over strata, and when correlation coefficients vary across strata. When the correlation coefficients are zero, close to zero, or the cell frequencies are small, the confidence intervals based on the restricted variance are preferred. For larger correlations and larger cell frequencies, the unrestricted confidence intervals give superior performance. We also apply the CMH statistic and proposed estimators to continuous non-normal data sampled from bivariate gamma distributions. We compare our methods to statistics for data sampled from normal distributions. The size and power of the CMH and normal theory statistics are comparable. When the stratum-specific sample sizes are small and the distributions are skewed, the proposed estimator is superior to the normal theory estimator. When the correlation coefficient is zero or close to zero, the restricted confidence intervals provide the best performance. None of the confidence intervals studied provides acceptable performances across all correlation coefficients, sample sizes and non-normal distributions.     

Sample size determination for multiple comparison studies treating confidence interval width as random.

Methods for optimal sample size determination are developed using four popular multiple comparison procedures (Scheffe's, Bonferroni's, Tukey's and Dunnett's procedures), where random samples of the same size n are to be selected from k (>/=2) normal populations with common variance sigma2, and where primary interest concerns inferences about a family of L linear contrasts among the k population means. For a simultaneous coverage probability of (1-alpha), the optimal sample size is defined to be the smallest integer value n*m such that, simultaneously for all L confidence intervals, the width of the lth confidence interval will be no greater than tolerance 2deltal (l=1,2,...,L) with tolerance probability at least (1-gamma), treating the pooled sample variance S2p as a random variable. Using Scheffe's procedure as an illustration, comparisons are made to usual sample size methods that incorrectly ignore the stochastic nature of S2p. The latter approach can lead to serious underestimation of required sample sizes and hence to unacceptably low values of the actually tolerance probability (1-gamma'). Our approach guarantees a lower bound of [1-(alpha+gamma)] for the probability that the L confidence intervals will both cover the parametric functions of interest and also be sufficiently narrow. Recommendations are provided regarding the choices among the four multiple comparison procedures for sample size determination and inference-making.     

An examination of methods for sample size recalculation during an experiment

In designing experiments, investigators frequently can specify an important effect that they wish to detect with high power, without the ability to provide an equally certain assessment of the variance of the response. If the experiment is designed based on a guess of the variance, an under-powered study may result. To remedy this problem, there have been several procedures proposed that obtain estimates of the variance from the data as they accrue and then recalculate the sample size accordingly. One class of procedures is fully sequential in that it assesses after each response whether the current sample size yields the desired power based on the current estimate of the variance. This approach is efficient, but it is not practical or advisable in many situations. Another class of procedures involves only two or three stages of sampling and recalculates the sample size based on the observed variance at designated times, perhaps coinciding with interim efficacy analyses. The two-stage approach can result in substantial oversampling, but it is feasible in many situations, whereas the three-stage approach corrects the problem of oversampling, but is less feasible. We propose a procedure that aims to combine the advantages of both the fully sequential and the two-stage approaches. This quasi-sequential procedure involves only two stages of sampling and it applies the stopping rule from the fully sequential procedure to data beyond the initial sample which we obtain via multiple imputation. We show through simulations that when the initial sample size is substantially less than the correct sample size, the mean squared error of the final sample size calculated from the quasi-sequential procedure can be considerably less than that from the two-stage procedure. We compare the distributions of these recalculated sample sizes and discuss our findings for alternative procedures, as well. © 1997 by John Wiley & Sons, Ltd.     

Sample size determinations using logistic regression with pilot data

Suppose the goal of a projected study is to estimate accurately the value of a ‘prediction’ proportion p that is specific to a given set of covariates. Available pilot data show that (1) the covariates are influential in determining the value of p and (2) their relationship to p can be modelled as a logistic regression. A sample size justification for the projected study can be based on the logistic model; the resulting sample sizes not only are more reasonable than the usual binomial sample size values from a scientific standpoint (since they are based on a model that is more realistic), but also give smaller prediction standard errors than the binomial approach with the same sample size. In appropriate situations, the logistic-based sample sizes could make the difference between a feasible proposal and an unfeasible, binomial-based proposal. An example using pilot study data of dental radiographs demonstrates the methods.     

Binary methods for continuous outcomes: a parametric alternative

Often a "disease" or "state of disease" is defined by a subdomain of a continuous outcome variable. For example, the subdomain of diastolic blood pressure greater than 90 mmHg has been used to define hypertension. The classical method of estimating the risk (or prevalence) of such defined disease states is to dichotomize the outcome variable according to the cutoff value. The standard statistical analysis of such risk of disease then exploits methods developed specifically for binary data, usually based on the binomial distribution. We present a method, based on the assumption of a Gaussian (normal) distribution for the continuous outcome, which does not resort to dichotomization. Specifically, the estimation of risk and its variance is presented for the one- and two-sample situations, with the latter focusing on risk differences and ratios, and odds ratios. The binomial approach applied to the dichotomized data is found to be less efficient than the proposed method by 67% or less. The latter is found to be very accurate, even for small sample sizes, although rather sensitive to substitutions of the underlying distribution by thicker tailed distributions. Canadian total cholesterol data are used to illustrate the problem. For the one-sample case, the approach is illustrated using data from a study of the arterial oxygenation of 20 patients during one-lung anesthesia for thoracic surgery. For the two-sample case, data from a prognostic study of the renal function of 87 lupus nephritic patients are used.     

Meta‐analysis of aggregate data on medical events

Meta‐analyses of clinical trials often treat the number of patients experiencing a medical event as binomially distributed when individual patient data for fitting standard time‐to‐event models are unavailable. Assuming identical drop‐out time distributions across arms, random censorship, and low proportions of patients with an event, a binomial approach results in a valid test of the null hypothesis of no treatment effect with minimal loss in efficiency compared with time‐to‐event methods. To deal with differences in follow‐up—at the cost of assuming specific distributions for event and drop‐out times—we propose a hierarchical multivariate meta‐analysis model using the aggregate data likelihood based on the number of cases, fatal cases, and discontinuations in each group, as well as the planned trial duration and groups sizes. Such a model also enables exchangeability assumptions about parameters of survival distributions, for which they are more appropriate than for the expected proportion of patients with an event across trials of substantially different length. Borrowing information from other trials within a meta‐analysis or from historical data is particularly useful for rare events data. Prior information or exchangeability assumptions also avoid the parameter identifiability problems that arise when using more flexible event and drop‐out time distributions than the exponential one. We discuss the derivation of robust historical priors and illustrate the discussed methods using an example. We also compare the proposed approach against other aggregate data meta‐analysis methods in a simulation study. Copyright © 2016 John Wiley & Sons, Ltd.     

Group sequential designs for three‐arm ‘gold standard’ non‐inferiority trials with fixed margin

In the recent years there have been numerous publications on the design and the analysis of three‐arm ‘gold standard’ noninferiority trials. Whenever feasible, regulatory authorities recommend the use of such three‐arm designs including a test treatment, an active control, and a placebo. Nevertheless, it is desirable in many respects, for example, ethical reasons, to keep the placebo group size as small as possible. We first give a short overview on the fixed sample size design of a three‐arm noninferiority trial with normally distributed outcomes and a fixed noninferiority margin. An optimal single stage design is derived that should serve as a benchmark for the group sequential designs proposed in the main part of this work. It turns out, that the number of patients allocated to placebo is substantially low for the optimal design. Subsequently, approaches for group sequential designs aiming to further reduce the expected sample sizes are presented. By means of choosing different rejection boundaries for the respective null hypotheses, we obtain designs with quite different operating characteristics. We illustrate the approaches via numerical calculations and a comparison with the optimal single stage design. Furthermore, we derive approximately optimal boundaries for different goals, for example, to reduce the overall average sample size. The results show that the implementation of a group sequential design further improves the optimal single stage design. Besides cost and time savings, the possible early termination of the placebo arm is a key advantage that could help to overcome ethical concerns. Copyright © 2013 John Wiley & Sons, Ltd.     

Power comparison of robust approximate and non-parametric tests for the analysis of cross-over trials

The main advantage of cross-over designs in practice is the use of a smaller number of subjects to produce treatment comparisons with sufficient precision. Bellavance and Tardif proposed a non-parametric approach to test the hypotheses of direct treatment and carry-over effects for the three-treatment three-period and six sequences cross-over design and showed the high asymptotic efficiency of their approach relative to the classical F-test based on ordinary least squares (OLS). In a more recent paper, Ohrvik suggested another non-parametric method for the analysis of cross-over trials. The power of these two non-parametric approaches is evaluated for small sample sizes via simulations, and compared to the power of the usual analysis of variance model based on OLS and a modified F-test approximation that take into account the correlation structure of the repeated measurements within subjects. Different covariance structures, sample sizes, and probability distributions for the responses, namely normal and gamma, are used in the simulations to evaluate the power and robustness of these different methods of analysis.     

Test statistics and sample size formulae for comparative binomial trials with null hypothesis of non-zero risk difference or non-unity relative risk

When it is required to establish a materially significant difference between two treatments, or, alternatively, to show that two treatments are equivalent, standard test statistics and sample size formulae based on a null hypothesis of no difference no longer apply. This paper reviews some of the test statistics and sample size formulae proposed for comparative binomial trials when the null hypothesis is of a specified non-zero difference or non-unity relative risk. Methods based on restricted maximum likelihood estimation are recommended and applied to studies of pertussis vaccine.     

Measuring variability between clusters by subgroup: An extension of the median odds ratio

Investigating clustered data requires consideration of the variation across clusters, including consideration of the component of the total individual variance that is at the cluster level. The median odds ratio and analogues are useful intuitive measures available to communicate variability in outcomes across clusters using the variance of random intercepts from a multilevel regression model. However, the median odds ratio cannot describe variability across clusters for different patient subgroups because the random intercepts do not vary by subgroup. To empower investigators interested in equity and other applications of this scenario, we describe an extension of the median odds ratio to multilevel regression models employing both random intercepts and random coefficients. By example, we conducted a retrospective cohort analysis of variation in care limitations (goals of care preferences) according to ethnicity in patients admitted to intensive care. Using mixed-effects logistic regression clustered by hospital, we demonstrated that patients of non-Caucasian ethnicity were less likely to have care limitations but experienced similar variability across hospitals. Limitations of the extended median odds ratio include the large sample sizes and computational power needed for models with random coefficients. This extension of the median odds ratio to multilevel regression models with random coefficients will provide insight into cluster-level variability for researchers interested in equity and other phenomena where variability by patient subgroup is important.     

A three-outcome design for randomized comparative phase II clinical trials

Randomized designs have been increasingly called for use in phase II oncology clinical trials to protect against potential patient selection bias. However, formal statistical comparison is rarely conducted due to the sample size restriction, despite its appeal. In this paper, we offer an approach to sample size reduction by extending the three-outcome design of Sargent et al. (Control Clin. Trials 2001; 22:117-125) for single-arm trials to randomized comparative trials. In addition to the usual two outcomes of a hypothesis testing (rejecting the null hypothesis or rejecting the alternative hypothesis), the three-outcome comparative design allows a third outcome of rejecting neither hypotheses when the testing result is in some 'grey area' and leaves the decision to the clinical judgment based on the overall evaluation of trial outcomes and other relevant factors. By allowing a reasonable region of uncertainty, the three-outcome design enables formal statistical comparison with considerably smaller sample size, compared to the standard two-outcome comparative design. Statistical formulation of the three-outcome comparative design is discussed for both the single-stage and two-stage trials. Sample sizes are tabulated for some common clinical scenarios.     

Early termination of a two‐stage study to develop and validate a panel of biomarkers

Two‐stage designs to develop and validate a panel of biomarkers present a natural setting for the inclusion of stopping rules for futility in the event of poor preliminary estimates of performance. We consider the design of a two‐stage study to develop and validate a panel of biomarkers where a predictive model is developed using a subset of the samples in stage 1 and the model is validated using the remainder of the samples in stage 2. First, we illustrate how we can implement a stopping rule for futility in a standard, two‐stage study for developing and validating a predictive model where samples are separated into a training sample and a validation sample. Simulation results indicate that our design has type I error rate and power similar to the fixed‐sample design but with a substantially reduced sample size under the null hypothesis. We then illustrate how we can include additional interim analyses in stage 2 by applying existing group sequential methodology, which results in even greater savings in the number of samples required under both the null and the alternative hypotheses. Our simulation results also illustrate that the operating characteristics of our design are robust to changes in the underlying marker distribution. Copyright © 2012 John Wiley & Sons, Ltd.