The power to detect differences in average rates of change in longitudinal studies

With considerable current interest in longitudinal epidemiologic studies, little is available regarding sample size requirements. This paper considers a method for analysis of longitudinal data, where one compares the mean rates of change for two or more groups, and proposes a statistic for use in determining sample size requirements. One calculates individual rates of change with least squares estimates of slopes of individuals' responses regressed over time. The assumption of linear change over time, while clearly not applicable for some data, applies to many biological measurements, either as recorded or with some transformation. The variances of these estimated slopes have two components: within-individual variability based on measurement error and length and frequency of follow-up, and true between-individual slope variability. It is assumed that measurement error is the same for all subjects, so that the total variances differ due to differences in follow-up. The question addressed is: when can one use the usual ANOVA F statistic to compare group means of estimated slopes? Expected mean squares demonstrate that this F is appropriate when either each group has the same number of subjects, or when each subject has the same length and frequency of follow-up. A procedure for computing power and sample size is presented, where one can specify the maximum detectable difference in any two average slopes. Moment estimation and maximum likelihood estimation of variance components from prior data are discussed.     

Covariate adjustment in randomized controlled trials with dichotomous outcomes increases statistical power and reduces sample size requirements

OBJECTIVE: Randomized controlled trials (RCTs) with dichotomous outcomes may be analyzed with or without adjustment for baseline characteristics (covariates). We studied type I error, power, and potential reduction in sample size with several covariate adjustment strategies. STUDY DESIGN AND SETTING: Logistic regression analysis was applied to simulated data sets (n=360) with different treatment effects, covariate effects, outcome incidences, and covariate prevalences. Treatment effects were estimated with or without adjustment for a single dichotomous covariate. Strategies included always adjusting for the covariate ("prespecified"), or only when the covariate was predictive or imbalanced. RESULTS: We found that the type I error was generally at the nominal level. The power was highest with prespecified adjustment. The potential reduction in sample size was higher with stronger covariate effects (from 3 to 46%, at 50% outcome incidence and covariate prevalence) and independent of the treatment effect. At lower outcome incidences and/or covariate prevalences, the reduction was lower. CONCLUSION: We conclude that adjustment for a predictive baseline characteristic may lead to a potentially important increase in power of analyses of treatment effect. Adjusted analysis should, hence, be considered more often for RCTs with dichotomous outcomes.     

A comparison of statistical methods for combining event rates from clinical trials

We compare two statistical methods for combining event rates from several studies. Both methods treat each study as a separate stratum. The Peto-modified Mantel-Haenszel (Peto) method estimates a combined odds ratio assuming homogeneity across strata and provides a test for heterogeneity. The DerSimonian and Laird modified Cochran method (D&L) produces a weighted average of rate differences, where the weights allow for among-study variability. We analyse 22 meta-analyses from ten reports by both methods. The pooled estimates are divided by their standard errors to produce a Z-statistic. A t-test comparing Z-statistics from all 22 studies suggests that the D&L method tends to be more conservative [d(Peto - D&L) = 0.29, t = 2.53, p = 0.02]. For a subset of 14 non-heterogeneous studies, the difference is smaller and non-significant (d = 0.09, t = 0.72, p = 0.49). The results from the methods correlate well (r = 0.66 for all 22 studies, r = 0.95 for 14 non-heterogeneous studies). Thus, the presence of heterogeneity influences our conclusion. We discuss the statistical and scientific implications of these findings.     

Ratio estimators of intervention effects on event rates in cluster randomized trials

We consider five asymptotically unbiased estimators of intervention effects on event rates in non-matched and matched-pair cluster randomized trials, including ratio of mean counts $\left(r_1\right)$, ratio of mean cluster-level event rates $\left(r_2\right)$, ratio of event rates $\left(r_3\right)$, double ratio of counts $\left(r_4\right)$, and double ratio of event rates $\left(r_5\right)$. In the absence of an indirect effect, they all estimate the direct effect of the intervention. Otherwise, $r_1$, $r_2,$ and $r_3$ estimate the total effect, which comprises the direct and indirect effects, whereas $r_4$ and $r_5$ estimate the direct effect only. We derive the conditions under which each estimator is more precise or powerful than its alternatives. To control bias in studies with a small number of clusters, we propose a set of approximately unbiased estimators. We evaluate their properties by simulation and apply the methods to a trial of seasonal malaria chemoprevention. The approximately unbiased estimators are practically unbiased and their confidence intervals usually have coverage probability close to the nominal level; the asymptotically unbiased estimators perform well when the number of clusters is approximately 32 or more per trial arm. Despite its simplicity, $r_1$ performs comparably with $r_2$ and $r_3$ in trials with a large but realistic number of clusters. When the variability of baseline event rate is large and there is no indirect effect, $r_4$ and $r_5$ tend to offer higher power than $r_1$, $r_2,$ and $r_3$. We discuss the implications of these findings to the planning and analysis of cluster randomized trials.     

Bayesian statistical inference enhances the interpretation of contemporary randomized controlled trials

ObjectiveRandomized trials generally use “frequentist” statistics based on P-values and 95% confidence intervals. Frequentist methods have limitations that might be overcome, in part, by Bayesian inference. To illustrate these advantages, we re-analyzed randomized trials published in four general medical journals during 2004.Study Design and SettingWe used Medline to identify randomized superiority trials with two parallel arms, individual-level randomization and dichotomous or time-to-event primary outcomes. Studies with P < 0.05 in favor of the intervention were deemed “positive”; otherwise, they were “negative.” We used several prior distributions and exact conjugate analyses to calculate Bayesian posterior probabilities for clinically relevant effects.ResultsOf 88 included studies, 39 were positive using a frequentist analysis. Although the Bayesian posterior probabilities of any benefit (relative risk or hazard ratio < 1) were high in positive studies, these probabilities were lower and variable for larger benefits. The positive studies had only moderate probabilities for exceeding the effects that were assumed for calculating the sample size. By comparison, there were moderate probabilities of any benefit in negative studies.ConclusionBayesian and frequentist analyses complement each other when interpreting the results of randomized trials. Future reports of randomized trials should include both.     

Alternatives to randomized controlled trials in telemedicine

Research designs other than true experiments may be useful in the evaluation of telemedicine. Potential methods include those that do not rely on randomization and tight control of the intervention, and analysis of existing administrative and clinical databases. Quasi-experimental designs may also be useful, especially when conducted in association with careful statistical methods that allow the investigator to control for certain differences between groups. Databases, such as those maintained by the Centers for Medicare and Medicaid Services, contain information on both outcomes and claims, as well as disease/procedure registries and electronic health records. This may provide a potential tool for understanding the effects of telemedicine on access to care in conjunction with costs and quality. These different approaches have advantages and disadvantages, but may be useful in telemedicine, where the conduct of randomized controlled trials is generally very expensive and frequently not feasible.     

Prospective adverse event risk evaluation in clinical trials

Health Care Management Science - Proactive and objective regulatory risk management of ongoing clinical trials is limited, especially when it involves the safety of the trial. We seek to...     

Estimating clinically significant differences in quality of life outcomes

Objective: This report extracts important considerations for determining and applying clinically significant differences in quality of life (QOL) measures from six published articles written by 30 international experts in the field of QOL assessment and evaluation. The original six articles were presented at the Symposium on Clinical Significance of Quality of Life Measures in Cancer Patients at the Mayo Clinic in April 2002 and subsequently were published in Mayo Clinic Proceedings. Principal findings: Specific examples and formulas are given for anchor-based methods, as well as distribution-based methods that correspond to known or relevant anchors to determine important differences in QOL measures. Important prerequisites for clinical significance associated with instrument selection, responsiveness, and the reporting of QOL trial results are provided. We also discuss estimating the number needed to treat (NNT) relative to clinically significant thresholds. Finally, we provide a rationale for applying group-derived standards to individual assessments. Conclusions: While no single method for determining clinical significance is unilaterally endorsed, the investigation and full reporting of multiple methods for establishing clinically significant change levels for a QOL measure, and greater direct involvement of clinicians in clinical significance studies are strongly encouraged.The Clinical Significance Consensus Meeting Group - See listing of members at the end of this article     

Late-starter sites in randomized controlled trials

In a cohort of 14 randomized controlled trials conducted by the Adult AIDS Clinical Trials Group between 1986 and 1999 with a target sample size of >400 (total enrollment 15,531 patients), we evaluated whether "late-starter" sites can make a meaningful contribution to eventual trial accrual. The sites that started recruiting within 5 months from the time the first patient entered the trial were eventually responsible for over 90% of the total enrollment in 11 of the 14 trials. Across the 14 trials, some sites were consistently among the first to start enrollment, whereas others were routinely among the last. The late-starter sites are unlikely to make important contributions to eventual trial enrollment in large clinical trials conducted by groups with a fixed number of sites. Protracting administrative efforts to add more sites many months after a multicenter trial has started may not be useful to trial accrual.     

Maximizing statistical power in group-randomized vaccine trials

Statistical power in group-randomized vaccine trials is complex: group randomization may increase power by capturing more transmission effects but may decrease power as more individuals are affected by a common source of variance. The former effect dominates when most infections arise from within the group and the latter when most arise outside. This process is complicated further when individuals possess partial natural immunity. Vaccine trials typically compare infection frequency (as measured by agent or antibody detection) in vaccinated vs. unvaccinated groups. To assess the relative contributions to statistical power by direct agent detection vs. serological detection of recent infection, we constructed stochastic compartmental models using differential equations describing all possible discrete states of the group. We contrasted models where natural immunity was absent, altered only the susceptible state, or altered both the susceptible and infected states. We observed the effects of endemic infection levels, the fraction of infection arising from outside the group, infectiousness vs. susceptibility vaccine effects and waning rates. There was significant enhancement of statistical power when serological testing separated infected and susceptible classes into subsets by natural immunity status.     

Monitoring potential adverse event rate differences using data from blinded trials: the canary in the coal mine

The development of drugs and biologicals whose mechanisms of action may extend beyond their target indications has led to a need to identify unexpected potential toxicities promptly even while blinded clinical trials are under way. One component of recently issued FDA rules regarding safety reporting requirements raises the possibility of breaking the blind for pre‐identified serious adverse events that are not the clinical endpoints of a blinded study. Concern has been expressed that unblinding individual cases of frequently occurring adverse events could compromise the overall validity of the study. However, if external information is available about adverse event rates among patients not receiving the test product in populations similar to the study population, then it may be possible to address the potential for elevated risk without unblinding the trial. This article describes a Bayesian approach for determining the likelihood of elevated risk suitable binomial or Poisson likelihoods that applies regardless of the metric used to express the difference. The method appears to be particularly appropriate for routine monitoring of safety information for project development programs that include large blinded trials. Copyright © 2016 John Wiley & Sons, Ltd.     

Exploring treatment preferences facilitated recruitment to randomized controlled trials

Objective

To explore how patients' treatment preferences were expressed and justified during recruitment to a randomized controlled trial (RCT) and how they influenced participation and treatment decisions.

Study Design and Setting

Qualitative analysis of audio recordings of recruitment appointments with 93 participants aged 51-70 years in a UK multicenter RCT of localized prostate cancer treatments.

Results

Treatment preferences at recruitment were more complex and dynamic than previously assumed. Most participants expressed views about treatments early in appointments, ranging on a continuum from hesitant to well-formed opinions. As recruiters elicited men’s views and provided detailed evidence-based treatment and study information, some opted for their preference, but many became uncertain and open to RCT recruitment, often accepting a different treatment from their original “preference.” Discussion of treatment preferences did not act as the expected barrier to recruitment but actively enabled many to express their concerns and reach an informed decision that often included RCT participation.

Conclusion

Exploring treatment preferences and providing evidence-based information can improve levels of informed decision making and facilitate RCT participation. Treatment preferences should be reconceptualized from a barrier to recruitment to an integral part of the information exchange necessary for informed decision making about treatments and RCT participation.     

Weight change and adverse event incidence with a low-dose oral contraceptive: two randomized, placebo-controlled trials

Changes in body weight and the incidence of estrogen-related side effects with low-dose oral contraceptives (OCs) containing 20 μg ethinyl estradiol (EE) have not been demonstrated in placebo-controlled trials. Two placebo-controlled, randomized trials demonstrated the efficacy of a low-dose OC for the treatment of acne in healthy females (n = 704; ≥14 years old) with regular menstrual cycles and moderate facial acne. Patients were randomized to receive 20 μg EE/100 μg levonorgestrel (LNG) or placebo for six cycles. Body weight was measured at baseline and during Cycles 1, 3, and 6. The occurrence of adverse events was recorded at each visit. Mean changes in weight from baseline were similar with 20 μg EE/100 μg LNG [0.72 kg ± 2.64 (SD; n = 349)] and placebo [0.56 kg ± 2.64 (SD; n = 355; p > 0.05)] for the last measured weight of each patient. Rates of headache, nausea, weight gain, and breast pain, side effects commonly attributed to OCs, were also similar between groups (p > 0.05). No serious, unexpected, drug-related adverse events occurred during the study. The low-dose OC containing 20 μg EE/100 μg LNG is safe, well tolerated, and does not cause weight gain.     

Robustness and power of analysis of covariance applied to ordinal scaled data as arising in randomized controlled trials

In clinical trials comparing two treatments, ordinal scales of three, four or five points are often used to assess severity, both prior to and after treatment. Analysis of covariance is an attractive technique, however, the data clearly violate the normality assumption and in the presence of small samples, and large sample theory may not apply. The robustness and power of various versions of parametric analysis of covariance applied to small samples of ordinal scaled data are investigated through computer simulation. Subjects are randomized to one of two competing treatments and the pre-treatment, or baseline, assessment is used as the covariate. We compare two parametric analysis of covariance tests that vary according to the treatment of the homogeneity of regressions slopes and the two independent samples t-test on difference scores. Under the null hypothesis of no difference in adjusted treatment means, we estimated actual significance levels by comparing observed test statistics to appropriate critical values from the F- and t-distributions for nominal significance levels of 0.10, 0.05, 0.02 and 0.01. We estimated power by similar comparisons under various alternative hypotheses. The model which assumes homogeneous slopes and the t-test on difference scores were robust in the presence of three, four and five point ordinal scales. The hierarchical approach which first tests for homogeneity of regression slopes and then fits separate slopes if there is significant non-homogeneity produced significance levels that exceeded the nominal levels especially when the sample sizes were small. The model which assumes homogeneous regression slopes produced the highest power among competing tests for all of the configurations investigated. The t-test on difference scores also produced good power in the presence of small samples.