Pharmacoeconomic evaluation in the real world. Effectiveness versus efficacy studies.

Pharmacoeconomic data may be obtained within the context of randomised clinical trials (RCTs) and from effectiveness studies in the 'real world'. The differences between the 2 types of study design have implications for the types of data that can be obtained and the interpretation of the resulting findings. Because RCTs are designed to assess the safety and efficacy of pharmaceuticals, and because the study design of RCTs emphasises internal validity over generalisability, the pharmacoeconomic data collected from them are limited. The data may not be applicable to the more heterogeneous patients encountered in actual clinical practice, and cost estimates may be inaccurate because of protocol requirements. Effectiveness studies, in which treatments are studied under real-world conditions, remedy some of these limitations. Generalisability to actual users is generally enhanced in effectiveness designs, but data may be biased in other ways. This brief review compares the 2 study designs as they relate to pharmacoeconomic evaluations in terms of the research questions they address, design differences and their implications for study bias, data collection and data analysis and the generalisability of their results.     

Use of randomised controlled trials for producing cost-effectiveness evidence: potential impact of design choices on sample size and study duration

BACKGROUND: A number of approaches to conducting economic evaluations could be adopted. However, some decision makers have a preference for wholly stochastic cost-effectiveness analyses, particularly if the sampled data are derived from randomised controlled trials (RCTs). Formal requirements for cost-effectiveness evidence have heightened concerns in the pharmaceutical industry that development costs and times might be increased if formal requirements increase the number, duration or costs of RCTs. Whether this proves to be the case or not will depend upon the timing, nature and extent of the cost-effectiveness evidence required. OBJECTIVE: To illustrate how different requirements for wholly stochastic cost-effectiveness evidence could have a significant impact on two of the major determinants of new drug development costs and times, namely RCT sample size and study duration. DESIGN: Using data collected prospectively in a clinical evaluation, sample sizes were calculated for a number of hypothetical cost-effectiveness study design scenarios. The results were compared with a baseline clinical trial design. RESULTS: The sample sizes required for the cost-effectiveness study scenarios were mostly larger than those for the baseline clinical trial design. Circumstances can be such that a wholly stochastic cost-effectiveness analysis might not be a practical proposition even though its clinical counterpart is. In such situations, alternative research methodologies would be required. For wholly stochastic cost-effectiveness analyses, the importance of prior specification of the different components of study design is emphasised. However, it is doubtful whether all the information necessary for doing this will typically be available when product registration trials are being designed. CONCLUSIONS: Formal requirements for wholly stochastic cost-effectiveness evidence based on the standard frequentist paradigm have the potential to increase the size, duration and number of RCTs significantly and hence the costs and timelines associated with new product development. Moreover, it is possible to envisage situations where such an approach would be impossible to adopt. Clearly, further research is required into the issue of how to appraise the economic consequences of alternative economic evaluation research strategies.     

真实世界研究与随机对照试验在临床实践及卫生决策中应用的比较

在临床指南和卫生政策制定的过程中，随机对照试验虽然一直被认为是"金标准"，但也存在外部效度较低的局限性，因此真实世界研究得到了迅速的发展。本文通过对国内外现有的相关研究和文献进行综述，对比真实世界研究与随机对照试验在临床实践和卫生决策应用中的差异，发现国内外现有关于真实世界研究与随机对照试验的研究仍然存在着很多问题，开展不同研究设计但针对同一临床问题相同干预措施的基于患者个体水平的临床研究，从方法学、结果、证据应用等多个层面对真实世界研究与随机对照试验进行系统比较，是十分必要与迫切的。     

临床适应性设计与药物评价的考虑

本文从药品评价的角度,对目前国际十分倡导的适应性设计阐述个人的认识,以及对一些问题的看法。合理地运用适应性设计不仅可以节省临床试验的研发时间和资金投入,而且也能为受试者带来利益。但是,如何保障适应性设计是在科学的基础上得到贯彻,取决于一系列的保障措施。目前形势下,应该是选择有代表性的品种,摸索和总结我国开展适应性设计的经验。适应性设计并不是新东西,而是我们对待临床设计变更的思想更新了。     

Cost effectiveness of cholinesterase inhibitors in the treatment of Alzheimer's disease: a review with methodological considerations

Cholinesterase inhibitors have been available for the treatment of Alzheimer's disease since 1993. They have significantly positive effects on cognitive functioning and other domains of functional capacity, such as activities of daily life in terms of efficacy, but the clinical value of these effects are under discussion. Cholinesterase inhibitors may also influence behavioural and psychological symptoms in Alzheimer's disease. Cholinesterase inhibitors are also regarded as rather expensive and, therefore, the question of cost effectiveness is essential. Pharmacoeconomic evaluations of cholinesterase inhibitors have so far been conducted in retrospect on efficacy data from prospective randomised clinical trials combined with economic data from other sources. There are no published specific cost-effectiveness studies of cholinesterase inhibitors which prospectively collected empirical data on costs and outcomes. There is only one published randomised clinical trial with such empirical data with a cost consequence analysis design, indicating cost neutrality. Several types of models to describe the long-term effects have been published, indicating cost effectiveness. However, due to methodological considerations, the validity of these models is difficult to judge. A research agenda for the cost effectiveness of cholinesterase inhibitors is proposed, including long-term studies with empirical data on resource use, costs and outcomes, studies on quality of life, informal care and behavioural and psychological symptoms, combination and comparative studies on mild cognitive impairment.     

Regulation of clinical trials in Europe

Pharmaceutical companies spend billions of dollars every year on carrying out clinical trials for their potential products. Many other trials are funded by charities, government research councils and other bodies. However, within Europe, national differences in approval procedures and the law relating to clinical trials can lead to increased costs and delays, particularly where trials are conducted in different countries. New European legislation is currently under review that is designed to ensure a common level of patient protection and scientific standards whilst reducing the costs and delays that can occur before trials can commence     

Operational Protocol Models in the Drug Development Process

Why do clinical trials that appear to be straightforward yield lower data quality and have longer timelines and higher costs than expected? Why do clinical development programs have difficulty combining studies when creating an integrated regulatory submission? Although the way these problems manifest themselves varies greatly, evidence continually points to the clinical trial protocol(s) as the root cause of operational errors, missed milestones, sponsor-site miscommunications, and incorrect data. Therefore, an up-front investment in analyzing the design features of a protocol for operational clarity and completeness could result in a substantial return from improved trial execution and regulatory reporting. A protocol design methodology, based on an innovative concept called the ‘operational protocol model’, has been developed to address operational problems. The operational protocol model replaces the current unstructured protocol document with a highly structured formal protocol model. A wide range of automated computer-based protocol quality analysis programs can utilize the operational protocol model to provide insight into protocol design from various perspectives. The operational protocol model is independent of the scientific, clinical, or statistical details of a study. Therefore, the operational protocol modeling methodology can be applied to markedly different protocol designs in widely differing clinical domains. Based on an empirical analysis of operational errors across a wide range of therapeutic areas, we have defined the following five key goals of a successful protocol design: coherency, completeness, consistency, comparability, and collaboration. The use of an operational protocol model can ensure that every clinical trial protocol designed within the operational protocol model meets all five protocol design quality goals.     

Is there a placebo problem in antidepressant trials?

In psychiatry, particularly in antidepressant clinical studies, placebo-controlled trials often yield results that are very difficult to interpret because of robust placebo responses. Meta-analyses of trials in major depressive disorder (MDD) suggest that drug-placebo differences in response rates range from 11% to 18%. However, in trials of marketed antidepressants present in the FDA databases, antidepressant drugs were superior to placebo in only 45 out of 93 RCTs (48%), and the placebo response overall appears to have increased over time. This gradual increase in placebo response rates may lead to delays in bringing new antidepressant treatments to the market, increased costs of antidepressant drug development and, in some cases, decisions to stop the development of certain compounds, or FDA decisions to not approve new treatments. A number of possible contributing factors to this significant placebo response in MDD have been identified, but further studies are needed. Many of the remedies used by researchers to minimize the placebo response, such as lead-in periods or shortening the duration of study visits, have failed to show consistent benefits. From our analysis of published studies, it appears that expectations about the speed of response may be shaped by the duration of the trial and that most of the placebo response occurs in the first half of the trial, regardless of its duration. These observations have led us to develop a novel approach to the placebo response problem called the Sequential Parallel Comparison Design.     

Prospective study designs in outcomes research: the case of migraine

Patient-reported outcomes (PROs), including resource utilisation, productivity and quality of life, are important outcomes in the field of migraine. Clinical trials have begun to incorporate PROs; however, not all research questions can be answered fully within the framework of a clinical trial design. Other prospective designs, including effectiveness trials, observational studies, and study hybrids may be used to answer many of the different research questions related to PROs. This paper reviews prospective study designs, their strengths and weaknesses, and examples of their application in migraine health-outcomes research. Guidance is provided for researchers in the selection of prospective research designs and the incorporation of PROs-research objectives.     

Using multiple drug exposure levels to optimize power in pharmacogenetic trials

Large-scale pharmacogenetic trials testing tens to hundreds of thousands of single-nucleotide polymorphisms (SNPs) will become possible in the near future given rapidly decreasing costs of genotyping. Devising optimal designs for these trials will be a significant challenge. The author demonstrates how the level of drug exposure may strongly affect the power to detect true associations between genetic polymorphisms and drug response. An analytic model of drug response is described that is used to simulate pharmacogenetic trials. Analytical and numerical sensitivity analyses are performed on the model to demonstrate possible exposure-sensitivity behaviors. This model shows that the power to detect an association can be a nonlinear, nonmonotonic function of drug exposure. This model is further investigated using two clinical trial designs. The conclusion is that trial designs that use more than one drug exposure or dose will have an increased likelihood of discovering statistically significant pharmacogenetic associations.     

Methodological and political issues in clinical pharmacology research by the year 2000

Parallel groups in a large, multicenter, phase III "pivotal" randomized clinical trial (RCT) with clinically relevant end-points are seen by the medical community as the "gold standard" of clinical research. However, there are limitations, some methodological and others political. The main one is the external validity of the method because a treatment, as studied in RCTs, does not necessarily reflect how it is used in clinical practice. Also, the method, as it stands, is not really predictive of the success in a particular patient of a certain intervention studied in a trial. To overcome these methodological drawbacks, different options have been implemented. The most important ones are: (1) the performance of pragmatic RCTs intended to address effectiveness rather than efficacy; (2) meta-analysis; and (3) the use of observational studies, with or without a comparison group. Recent experience has shown that type-A adverse drug reactions (ADRs) related to a specific class of drugs have been successfully characterized throughout cohort- or population-based case-control studies, whereas the evidence linking a specific drug entity to a type-B ADR, apparently severe enough to withdraw the drug from the market, has come mainly from case reports or case series. Other limitations of the RCT are more of a political nature. These large "pivotal" trials are mostly sponsored by the pharmaceutical industry, and to guarantee the scientific and ethical integrity of data produced, they are performed following standard operating procedures (SOPs) and good clinical practice (GCP) guidelines. Sometimes industry is not interested in sponsoring trials; thus, RCTs performed are in practice highly biased because of their potential economical profits. Furthermore, applying SOPs and GCPs is expensive and difficult to implement, and it is hard to find funding in public institutions. As a result, there is an urgent need to create a network of independent, skilled groups interested in sponsoring and performing institutional RCTs following "user friendly" GCP when the profits are low, but scientific interest high.     

Translating research into clinical practice: critical interpretation of clinical trials in schizophrenia

Translating research into clinical practice has in part been hampered by practitioners' difficulties in interpreting statistical measures. The main part of this article aims to bridge the gap between research and practice by describing frequently occurring problems in the methodology and reporting of clinical trials. Basic methodological concepts and statistical measures, such as different trial designs, randomization, blinding, case numbers, eligibility criteria, internal and external validity, efficacy versus effectiveness, the meaning of rating scales, choice of comparators, effect size versus P-value and the interpretation of risk measures, are explained and their use illustrated. The text further presents the outcome from a workshop on 'Translating Clinical Studies into Practice' held at an international conference on schizophrenia and bipolar disorder in June 2005. The participants suggested more emphasis on additional secondary endpoints in clinical trials such as patients' self assessments and indicators of well-being. Better training of clinicians in the interpretation of scientific results and the use of more meaningful clinical outcomes are both needed to effectively translate research into clinical practice.     

Medication screening for smoking cessation: a proposal for new methodologies

Rationale The purpose of medication screening studies is to quickly and cheaply evaluate the clinical potential of medications, so that promising drugs proceed to large-scale clinical trials and unpromising drugs do not. Screening procedures for smoking cessation medications either are not sufficiently practical or lack clinical validity. Clinical trials have clinical validity but are often impractical as initial tests of efficacy (i.e., screening) or suffer from limited statistical power. The alternative approach of short-term, laboratory-based studies of purported mechanisms of efficacy may overcome some of the practical problems of clinical trials but appear to have limited clinical validity. Objectives This commentary identifies some of the limitations of current short-term screening procedures and provides suggestions for improving such studies. Results Short-term screening studies typically use smokers unmotivated to abstain (i.e., nontreatment seekers) as participants and examine brief medication effects on clinical markers or potential mechanisms of action, including relief of withdrawal and craving during enforced abstinence or on reduction in the reinforcing effects of smoked tobacco. The limitation of these approaches is shown by their insensitivity to effects of nicotine replacement and bupropion, which are effective in clinical trials for smoking cessation. Conclusions The clinical validity of short-term screening studies may improve if these studies simulate some clinical trial procedures within practical limitations. Thus, they should recruit smokers motivated to abstain, emphasize smoking abstinence as a primary index of medication response, examine effects over sufficiently long time periods to encompass the drug's mechanism of action, and assess responses in the natural environment. Whether these changes improve the sensitivity of screening studies is testable. Other research aimed specifically at identifying the mechanisms of therapeutic action of a medication may also profit from using this approach of simulating a short-term clinical trial.     

A pathway to improved prospective observational post-authorization safety studies

Randomized controlled trials (RCTs) are the gold standard for assessing the efficacy of drugs but not necessarily so for drug safety where inadequate power to detect either multiple or rare adverse events is a major handicap. Furthermore, the conditions under which drugs are approved for market use are often different from the settings in actual use. Indeed, with their control mechanisms, trials are by design largely inadequate for the identification of potential safety signals, especially of the rare type, hence the value of postmarketing surveillance and risk management plan-based activities. Today, clinical trials constitute only a part of the research that goes into assessing the safety of drugs. Observational studies, where the investigators merely collect data on treatments received by patients and their health status in routine clinical practice are increasing in uptake because they reflect the real-life utility of drugs, despite the absence of random treatment assignment. Although such studies generally provide less compelling evidence than RCTs, they can be far more useful to drug safety assessment activities than generally acknowledged. An increasing number of post-authorization safety studies (PASS) within the European Medicines Agency's jurisdiction are of the observational type - considered perhaps as more appropriate vehicles for exploring and documenting how products perform in the real world. A similar trend is emerging in the US following the FDA Amendments Act of 2007; since early 2010, an increasing number of post-approval commitments mandated by the FDA include observational studies. However, despite this pattern, not much is known about ongoing efforts to address many of the recognized inadequacies associated with existing methodologies and practices currently adopted in observational PASS. This current opinion presents an overview of some of the main challenges we face in prospective observational PASS, mainly from practical experience, and proposes certain steps for improvement.     

Modelling and simulation of placebo effect: application to drug development in schizophrenia

High and variable placebo effect (PE) within and among clinical trials can substantially affect conclusions about the efficacy of new drugs in the treatment of schizophrenia and other neuropsychiatric disorders. In recent years, it has become increasingly difficult to prove drug efficacy against placebo, and one of the reasons is that the placebo response has increased over recent years. The increased placebo response over the years is partly explained by unidentified parallel interventions, patient factors, issues with trial designs, and regional variability or demographic differences. In addition, a nocebo effect, which is undesirable effects a subject manifests after receiving placebo, e.g. extrapyramidal side effects, in placebo arms of antipsychotic trials could also influence the PE and clinical trial outcomes. Placebo effects (PEs) are a natural phenomenon and cannot be avoided completely in clinical trials. However, accounting for the PE via mixed effects modelling approaches could reduce bias in quantifying the overall effect size of the drug treatment. This review article focuses on the PE and its impact on schizophrenia clinical trial outcomes. The authors briefly describe the factors that lead to high and variable PE. Next, pharmacometric approaches to account for the PE and dropouts in schizophrenia clinical trials are described. Finally, some points are provided that could be considered while designing and optimizing antipsychotic trials via simulation approaches.     

Ranking evidence in substance use and addiction

Evidence-based medicine has consistently prized the epistemological value of randomized-controlled trials (RCTs) owing to their methodological advantages over alternative designs such as observational studies. However, there are limitations to RCTs that hinder their ability to study chronic and dynamic conditions such as substance use and addiction. For these conditions, observational studies may provide superior evidence based on methodological and practical strengths. Assuming epistemic superiority of RCTs has led to an inappropriate devaluation of other study designs and the findings they support, including support for harm reduction services, especially needle exchange programs and supervised injection facilities. The value offered by observational studies should be reflected in evidence-based medicine by allowing more flexibility in evidence hierarchies that presume methodological superiority of RCTs. Despite the popularity of evidence ranking systems and hierarchies, nothing should replace critical appraisal of study methodology and examining the suitability of applying a given study design to a specific research question.     

An Adaptive Group Sequential Phase II Design to Compare Treatments for Survival Endpoints in Rare Patient Entities

For rare diseases, standard treatments are often not available and essential study parameters are difficult or impossible to obtain. Therefore, designs of clinical trials for these diseases are often based on little information. Adaptive designs allow such trials to be started and to gain information during the study. Motivated by a trial for a rare subtype of renal-cell carcinoma, we present a two-stage adaptive design for right-censored time-to-event data and a two-sided test. After the first stage, one can stop for futility or continue with reestimated sample size. The properties of such designs are analyzed by simulation studies.     

Single versus multiple drug focus in substance abuse clinical trials research

Complex patterns of multiple substance use pose clinical and methodological challenges for substance abuse clinical trials research. To increase measurement precision and internal validity, the modal approach has been to target both treatment interventions and outcome assessment to a single class of abused substance. This strategy warrants reconsideration because it entails limitations in recruitment feasibility and generalization of study findings. This report reviews pros and cons of single versus multiple targeted drugs, suggests guidelines for choosing between these strategies and outlines methods for broadening the scope of substance abuse clinical trails to take abuse of multiple substances into account. We recommend that investigators consider moving away from a single drug focus in three ways. First, include systematic assessment of a wide range of psychoactive substance use throughout the trial and evaluate the impact of study treatments on use of all classes of drugs. Second, except where contraindicated, include patients who use and abuse multiple classes of substances even in trials evaluating treatment of a single targeted drug. Third, consider inclusion of polysubstance abusers or those who primarily abuse multiple classes of substances in the same clinical trial. Although many treatment efficacy questions can best be answered by single focus studies, we recommend that such designs be adopted only after less restrictive designs are first considered.     

An adaptive group sequential phase II design to compare treatments for survival endpoints in rare patient entities

For rare diseases, standard treatments are often not available and essential study parameters are difficult or impossible to obtain. Therefore, designs of clinical trials for these diseases are often based on little information. Adaptive designs allow such trials to be started and to gain information during the study. Motivated by a trial for a rare subtype of renal-cell carcinoma, we present a two-stage adaptive design for right-censored time-to-event data and a two-sided test. After the first stage, one can stop for futility or continue with reestimated sample size. The properties of such designs are analyzed by simulation studies.