Randomized consent designs for clinical trials: an update

Randomized consent designs were introduced to make it easier for physicians to enter patients in randomized clinical trials. Physician reluctance to participate in randomized clinical trials is often a reflection that the physician-patient relationship could be compromised if the physician makes known to the patient his/her inability to select a preferred therapy. Clinical trials having a no-treatment control or placebo amplify this concern. This paper reviews the main ideas of randomized consent designs (single and double) and the statistical model underlying the analysis, and presents some recent experiences.     

Evaluating medication effects outside of clinical trials: new-user designs

Recent clinical trials demonstrating that hormone replacement therapy (HRT) does not prevent coronary heart disease in women have again raised doubts concerning observational studies. Although much of the explanation probably lies in what might be called the "healthy HRT user" effect, another contributing factor may be that most observational studies included many prevalent users: women taking HRT for some time before study follow-up began. This practice can cause two types of bias, both of which plausibly may have contributed to the discrepancy between observational and randomized studies. First, prevalent users are "survivors" of the early period of pharmacotherapy, which can introduce substantial bias if risk varies with time, just as in studies of operative procedures that enroll patients after they have survived surgery. This article provides several examples of medications for which the hazard function varies with time and thus would be subject to prevalent user bias. Second, covariates for drug users at study entry often are plausibly affected by the drug itself. Investigators often do not adjust for these factors on the causal pathway, which may introduce confounding. A new-user design eliminates these biases by restricting the analysis to persons under observation at the start of the current course of treatment. This article thus argues that such designs should be used more frequently in pharmacoepidemiology.     

A measure to aid in the interpretation of published clinical trials

The published literature contains numerous reports of clinical studies. A problem in their interpretation is that studies in which the observed efficacy of the treatment is high are much more likely to be reported than those in which the observed efficacy is average or poor. This phenomenon has had the consequence of generally discrediting the reliability of the literature, especially that of non-randomized studies. In this paper a model is developed which permits estimation of the potential magnitude by which the reported efficacy of a treatment might be inflated. This quantity is termed the publication bias. The magnitude of the bias depends on the sample size of the study and the number of similar studies conducted concurrently. Tabulated values of the bias are presented, permitting easy computation. The new measure may have potential use for physicians in clinical decision making in that it characterizes the reliability of results from a specific published study, especially when there are no definitive randomized studies. However, correction of publication bias in this manner is not a substitute for a well controlled or a randomized study. The technique merely assists in the interpretation of available evidence from the literature. Moreover, it must be used with due caution in recognition of the assumptions and approximations involved in the calculation.     

Weak associations in epidemiological research: some examples and their interpretation

The study of the epidemiology of chronic non-infectious diseases grew rapidly after the Second World War. In the early 1950s strong associations were found in studies based on hypotheses derived from clinical, demographic and animal observations. The associations between smoking and lung cancer, physical exercise and myocardial infarction, and air pollution and mortality all led to more than three decades of epidemiological endeavour to refine our understanding of the initial observations. As the large effects were discovered, new associations were perforce of smaller magnitude. The benefits of milk supplements to the diet of school-age children was elegantly shown in an MRC trial in 1920s using small numbers of deprived children, whereas in the 1970s, when the population's nutrition was vastly improved, many thousands of children were needed to give an answer. The impact of passive smoking on cancer incidence, on children's health and on pregnancy is a current debate because of the vanishingly small effects. Studies of the relation between alcohol consumption during pregnancy and the outcome of pregnancy has given rise to conflicting findings so that clear cut scientifically based recommendations cannot be given to social drinkers. The demands of governments for epidemiological evidence on which to base standards for pollutants in the air, water and ground, has resulted in the need for multiple studies using different techniques to suggest whether or not an association between health and exposure exists. These examples are used to illustrate the difficulties of interpretation of epidemiological studies when the effects of suspected risk factors are small.     

Formal approaches to safety monitoring of clinical trials in life-threatening conditions

Large clinical trials in life-threatening conditions are usually conducted under the surveillance of a Data and Safety Monitoring Board (DSMB), whose remit is to protect the ethical and safety interests of the patients. The purpose of this paper is to describe a formal approach to safety monitoring, using a sequential safety procedure to aid the decisions made by the DSMB. This procedure is designed to recommend termination of the study as soon as evidence that the experimental treatment is worse than the control in terms of the primary safety response is so strong that it is unethical to proceed. The use of this formal sequential procedure enables probabilities of the study stopping erroneously and stopping correctly, under various degrees of experimental treatment disadvantage, to be examined. Also scenarios depicting data sets which lead to continuing or stopping can be presented. Such explorations are useful in encouraging all DSMB members to consider carefully, prior to the start of a study, the conditions under which they would seriously wish to consider termination. The implementation of these methods is described for three recently completed trials in which it has been used. Finally, our current recommendations for the design of these procedures, arising from these and other similar experiences, are given.     

Optimal approaches to data collection and analysis of potential immune mediated disorders in clinical trials of new vaccines

Background

The potential for development of autoimmune diseases after vaccination with new vaccines containing novel adjuvants is a theoretical concern. Randomised, placebo-controlled trials are the best method for assessing a potential causal relationship between an adverse event and vaccination, but usually have a sample size too small to detect adverse events occurring in <1% of subjects. Incomplete case documentation may hamper definitive diagnoses, preventing accurate causality assessment. To date there are no guidelines for collection, documentation and monitoring of potential immune mediated disorders (pIMD) reported in the course of clinical trials with adjuvanted vaccines.

Objective

This paper proposes a methodology for collection of pIMDs in clinical vaccine trials, with the objective of obtaining complete and reliable data using standardised methodology for its collection and analysis.

Recommendations

The role of the study investigator in prospective, standardised safety data collection is key and can be facilitated by providing a pIMD list in study documents and disease-specific standard questionnaires to assist timely and thorough documentation. External expert review of histopathology samples or other specialised diagnostic data would increase diagnostic accuracy. Centralised case ascertainment using standard case definitions would identify true cases of interest. We propose collection of safety data for at least 6 months and up to one year after the last vaccine dose. Bio-banking as a platform for collecting samples from enrolled patients for future use (e.g., to measure biomarkers of diagnostic, prognostic or predictive utility) could eventually provide valuable information in cases where a pIMD is diagnosed during the study period.

Conclusion

Standardised collection of safety data to allow appropriate analyses are optimal approaches for detecting rare events in clinical trials. Appropriate data analysis will then more reliably define potential causal relationships with vaccination.     

Interim analyses in clinical trials: Classical vs. bayesian approaches

This paper concerns interim analysis in clinical trials involving two treatments from the points of view of both classical and Bayesian inference. I criticize classical hypothesis testing in this setting and describe and recommend a Bayesian approach in which sampling stops when the probability that one treatment is the better exceeds a specified value. I consider application to normal sampling analysed in stages and evaluate the gain in average sample number as a function of the number of interim analyses.     

Multicentre clinical trials: Bradford Hill's contributions and some subsequent developments

Most large comparative trials of therapeutic agents are now conducted as multicentre studies. Some of the major studies designed by Bradford Hill were also multicentre, and his methods of design and conduct remain valid today, even though improved methods for analysing the data have appeared subsequently.     

Randomized trials with concurrent economic evaluations reported unrepresentatively large clinical effect sizes

OBJECTIVE: To examine whether randomized economic evaluations report clinical effectiveness estimates that are unrepresentative of the totality of the research literature. STUDY DESIGN AND SETTING: From 36 studies (12,294 patients) of enhanced care for depression, we compared pooled clinical effect sizes in studies with a concurrent economic evaluation to those in studies that did not publish a concurrent economic evaluation, using metaregression. RESULTS: The pooled clinical effect size of studies publishing an economic evaluation was almost twice as large as that of studies that did not publish an economic evaluation (pooled standardized mean difference [SMD] in randomized controlled trials [RCTs] with an economic evaluation=0.34; 95% confidence interval [CI]=0.23-0.46; pooled SMD in RCTs without an economic evaluation=0.17; 95% CI=0.10-0.25). This difference was statistically significant (SMD between group difference=-0.17; 95% CI: -0.31 to -0.02; P=0.02). CONCLUSION: Publication of an economic evaluation of enhanced care for depression was associated with a larger clinical effect size. Cost-effectiveness estimates should be interpreted with caution, and the representativeness of the clinical data on which they are based should always be considered. Further research is needed to explore this observed association and potential bias in other areas.     

Group comparisons involving missing data in clinical trials: a comparison of estimates and power (size) for some simple approaches

When using 'intent-to-treat' approaches to compare outcomes between groups in clinical trials, analysts face a decision regarding how to account for missing observations. Most model-based approaches can be summarized as a process whereby the analyst makes assumptions about the distribution of the missing data in an attempt to obtain unbiased estimates that are based on functions of the observed data. Although pointed out by Rubin as often leading to biased estimates of variances, an alternative approach that continues to appear in the applied literature is to use fixed-value imputation of means for missing observations. The purpose of this paper is to provide illustrations of how several fixed-value mean imputation schemes can be formulated in terms of general linear models that characterize the means of distributions of missing observations in terms of the means of the distributions of observed data. We show that several fixed-value imputation strategies will result in estimated intervention effects that correspond to maximum likelihood estimates obtained under analogous assumptions. If the missing data process has been correctly characterized, hypothesis tests based on variances estimated using maximum likelihood techniques asymptotically have the correct size. In contrast, hypothesis tests performed using the uncorrected variance, obtained by applying standard complete data formula to singly imputed data, can provide either conservative or anticonservative results. Surprisingly, under several non-ignorable non-response scenarios, maximum likelihood based analyses can yield equivalent hypothesis tests to those obtained when analysing only the observed data.     

Randomized clinical trials and observational studies in the assessment of drug safety

Randomized clinical trials are considered as the preferred design to assess the potential causal relationships between drugs or other medical interventions and intended effects. For this reason, randomized clinical trials are generally the basis of development programs in the life cycle of drugs and the cornerstone of evidence-based medicine. Instead, randomized clinical trials are not the design of choice for the detection and assessment of rare, delayed and/or unexpected effects related to drug safety. Moreover, the highly homogeneous populations resulting from restrictive eligibility criteria make randomized clinical trials inappropriate to describe comprehensively the safety profile of drugs. In that context, observational studies have a key added value when evaluating the benefit-risk balance of the drugs. However, observational studies are more prone to bias than randomized clinical trials and they have to be designed, conducted and reported judiciously. In this article, we discuss the strengths and limitations of randomized clinical trials and of observational studies, more particularly regarding their contribution to the knowledge of medicines’ safety profile. In addition, we present general recommendations for the sensible use of observational data.     

Novel approaches to incorporating pharmacoeconomic studies into phase III clinical trials for Alzheimer's disease

The societal and individual costs of Alzheimer's disease are significant, worldwide. As the world ages, these costs are increasing rapidly, while health systems face finite budgets. As a result, many regulators and payers will require or at least consider phase III cost-effectiveness data (in addition to safety and efficacy data) for drug approval and reimbursement, increasing the risks and costs of drug development. Incorporating pharmacoeconomic studies in phase III clinical trials for Alzheimer's disease presents a number of challenges. We propose several specific suggestions to improve the design of pharmacoeconomic studies in phase III clinical trials. We propose that acute episodes of care are key outcome measures for pharmacoeconomic studies. To improve the possibility of detecting a pharmacoeconomic impact in phase III, we suggest several strategies including; study designs for enrichment of pharmacoeconomic outcomes that include co-morbidity of patients; reducing variability of care that can affect pharmacoeconomic outcomes through standardized care management; employing administrative claims data to better capture meaningful pharmacoeconomic data; and extending clinical trials in open label follow-up periods in which pharmacoeconomic data are captured electronically by administrative claims. Specific aspects of power analysis for pharmacoeconomic studies are presented. The particular pharmacoeconomic challenges caused by the use of biomarkers in clinical trials, the increasing use of multinational studies, and the pharmacoeconomic challenges presented by biologicals in development for Alzheimer's disease are discussed. In summary, since we are entering an era in which pharmacoeconomic studies will be essential in drug development for supporting regulatory approval, payor reimbursement and integration of new therapies into clinical care, we must consider the design and incorporation of pharmacoeconomic studies in phase III clinical trials more seriously and more creatively.     

Modelling to extract more information from clinical trials data: On some roles for the bootstrap

Despite its importance in the theoretical literature, the bootstrap appears to play a negligible role in pharmaceutical research, as will be demonstrated by a brief literature review. As will be shown by examples, the bootstrap is a useful tool in the planning and analysis of clinical trials. The first example shows that some important information required in the design of a study can best be gained by using the bootstrap. It is argued from two further examples that more information can be extracted from large clinical trials by data-dependent modelling. This is shown by identifying a prognostic factor that may play a role as an inclusion criterion of a new study and by an interaction of a continuous predictor with treatment. To protect against erroneous conclusions from data-dependent modelling in a multivariable context, detailed checks of the results and stability analyses should be performed by the bootstrap. To conclude, a discussion of the future of modelling and the future of the bootstrap is given.     

Generalizability analysis for clinical trials: a simulation study

Subjects are rarely selected on a random basis from a well‐defined patient population of interest into a clinical trial, with women, children, the elderly, and those with common comorbidities who are frequently underrepresented. Decades of clinical experience have demonstrated that the application of trial findings to individual patients is permissible by using efficacy as a measure of effectiveness and assuming that the characteristics of patients are sufficiently similar. In order to investigate this issue in greater depth, we simulated a patient population with treatment effect size of 0.5 (Cohen's d) and five covariates that included gender, health insurance, comorbidity, age, and motivation. To demonstrate how selection of patients for a clinical trial can bias the results when treatment effect varies across individuals, we created 50 nonrandom clinical trials based on this patient population and showed relative bias to range from 1.68% to 99.70%. We calculated and evaluated three indexes: C‐statistics, standardized mean difference (SMD), and Tipton's index (β) of generalization for the 50 nonrandom trials. Findings indicated that (i) the ranges were 0.56–0.98, 0.23–11.17, and 0.99–0.73 for C‐statistics, SMD, and β, respectively, when treatment effect bias increased from 1.68% to 99.70% and (ii) C‐statistics < 0.86, SMD < 1.95, and β > 0.91 when treatment effect bias <50%. Recommendations are made using existing generalization indexes on the basis of our simulation results. An example from a real clinical trial is provided for illustration. Copyright © 2017 John Wiley & Sons, Ltd.     

Adjusting for drop-out in clinical trials with repeated measures: design and analysis issues

Recently, Wu and Follmann developed summary measures to adjust for informative drop-out in longitudinal studies where drop-out depends on the underlying true value of the response. In this paper we evaluate these procedures in the common situation where drop-out depends on the observed responses. We also discuss various design and analysis strategies which minimize the bias obtained with this type of drop-out. Of particular interest is the use of multiple measurements of the response at each visit to reduce bias. These strategies are evaluated with a simulation study. The results are highlighted with applications to both a hypertensive and a respiratory disease clinical trial, where multiple measurements of the primary response were made for all participants at each visit.     

Coping with missing data in clinical trials: a model-based approach applied to asthma trials

In most clinical trials, some patients do not complete their intended follow-up according to protocol, for a variety of reasons, and are often described as having 'dropped out' before the conclusion of the trial. Their subsequent measurements are missing, and this makes the analysis of the trial's repeated measures data more difficult. In this paper we briefly review the reasons for patient drop-out, and their implications for some commonly used methods of analysis. We then propose a class of models for modelling both the response to treatment and the drop-out process. Such models are readily fitted in a Bayesian framework using non-informative priors with the software BUGS. The results from such models are then compared with the results of standard methods for dealing with missing data in clinical trials, such as last observation carried forward. We further propose the use of a time transformation to linearize an asymptotic pattern of repeated measures over time and therefore simplify the modelling. All these ideas are illustrated using data from a five-arm asthma clinical trial.