Constructing Experimental Designs for Discrete-Choice Experiments: Report of the ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force

Stated-preference methods are a class of evaluation techniques for studying the preferences of patients and other stakeholders. While these methods span a variety of techniques, conjoint-analysis methods—and particularly discrete-choice experiments (DCEs)—have become the most frequently applied approach in health care in recent years. Experimental design is an important stage in the development of such methods, but establishing a consensus on standards is hampered by lack of understanding of available techniques and software. This report builds on the previous ISPOR Conjoint Analysis Task Force Report: Conjoint Analysis Applications in Health—A Checklist: A Report of the ISPOR Good Research Practices for Conjoint Analysis Task Force. This report aims to assist researchers specifically in evaluating alternative approaches to experimental design, a difficult and important element of successful DCEs. While this report does not endorse any specific approach, it does provide a guide for choosing an approach that is appropriate for a particular study. In particular, it provides an overview of the role of experimental designs for the successful implementation of the DCE approach in health care studies, and it provides researchers with an introduction to constructing experimental designs on the basis of study objectives and the statistical model researchers have selected for the study. The report outlines the theoretical requirements for designs that identify choice-model preference parameters and summarizes and compares a number of available approaches for constructing experimental designs. The task-force leadership group met via bimonthly teleconferences and in person at ISPOR meetings in the United States and Europe. An international group of experimental-design experts was consulted during this process to discuss existing approaches for experimental design and to review the task force’s draft reports. In addition, ISPOR members contributed to developing a consensus report by submitting written comments during the review process and oral comments during two forum presentations at the ISPOR 16th and 17th Annual International Meetings held in Baltimore (2011) and Washington, DC (2012).     

Good Research Practices for Measuring Drug Costs in Cost-Effectiveness Analyses: An International Perspective: The ISPOR Drug Cost Task Force Report—Part VI

Objective:  The pharmacoeconomic guidelines available in the literature or promulgated in many countries are either vague or silent about how drug costs should be established or measured so an international comparison of cost-effectiveness analysis (CEA) results can be made. The objective of this report is to provide guidance and recommendations on how drug costs should be measured for CEAs done from an internationally comparative perspective.
Methods:  Members of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Task Force on Good Research Practices—Use of Drug Costs for Cost Effectiveness Analysis (Drug Cost Task Force [DCTF]) subgroup from several countries were experienced developers or users of CEA models, and worked in academia, industry, and as advisors to governments. They solicited comments on drafts from a core group of 174 external reviewers and more broadly, from the members of the ISPOR at the ISPOR 12th Annual International meeting and via the ISPOR Web site.
Results:  Drug units should be standardized in terms of volume of active ingredient, regardless of packaging and dosing strength variations across countries. Drug costs should be measured in local currency per unit of active ingredient and should be converted to other currencies using sensitivity analyses of purchasing power parities (PPP) and exchange rates, whichever is more appropriate. When using drug prices from different years, the consumer price index for the local currency should be applied before the PPP and/or exchange rate conversion.
Conclusion:  CEA researchers conducting international pharmacoeconomic analysis should tailor the appropriate measure of drug costs to the international perspective, to maintain clarity and transparency on drug cost measurement in the context of international drug comparison and report the sensitivity of CEA results to reasonable cost conversions.     

Good Research Practices for Measuring Drug Costs in Cost Effectiveness Analyses: An Industry Perspective: The ISPOR Drug Cost Task Force Report—Part V

Objectives:  The industry perspective on drug costs should be framed by the need for decision-makers to use actual and relevant costs, and to inform real-world decisions regarding medication selection and use. The objective of this report is to provide guidance and recommendations on how manufacturers should approach the use of drug costs.
Methods:  The Task Force was appointed with the advice and consent of the ISPOR Board of Directors. Members were experienced developers or users of drug cost information working in academia and industry, and came from several countries. Following the core assumptions developed and outlined by the Task Force, a draft report was prepared. Comments were solicited on the outline and several draft reports both from a core group of external reviewers and more broadly from the ISPOR membership of ISPOR via the ISPOR Web site.
Results:  The industry should always strive for: 1) a focus on drug value and not just cost; 2) credibility—that is correct and consistent costs; 3) transparency—by disclosing the prices and costs, and ensuring that they reflect the actual cost of the drug whenever possible; and 4) providing actionable results that help customers comprehend the value offered by a drug therapy and to use products more efficiently and effectively.
Conclusions:  Understanding and accounting for all costs and consequences of the use of a medical treatment is in the best interests of all parties involved in the prescribing, consuming, reimbursement, selling, and manufacturing of bio/pharmaceuticals. Transparency, consistency, and clear communication of costs and value are essential for appropriate decision-making and should be important goals for all parties.     

Using Propensity Scores for Causal Inference: Pitfalls and Tips

Methods based on propensity score (PS) have become increasingly popular as a tool for causal inference. A better understanding of the relative advantages and disadvantages of the alternative analytic approaches can contribute to the optimal choice and use of a specific PS method over other methods. In this article, we provide an accessible overview of causal inference from observational data and two major PS-based methods (matching and inverse probability weighting), focusing on the underlying assumptions and decision-making processes. We then discuss common pitfalls and tips for applying the PS methods to empirical research and compare the conventional multivariable outcome regression and the two alternative PS-based methods (ie, matching and inverse probability weighting) and discuss their similarities and differences. Although we note subtle differences in causal identification assumptions, we highlight that the methods are distinct primarily in terms of the statistical modeling assumptions involved and the target population for which exposure effects are being estimated.Key words: propensity score, matching, inverse probability weighting, target population     

Good Research Practices for Measuring Drug Costs in Cost-Effectiveness Analyses: A Managed Care Perspective: The ISPOR Drug Cost Task Force Report—Part III

Objectives:  The objective of this report is to provide guidance and recommendations on how drug costs should be measured for cost-effectiveness analyses conducted from the perspective of a managed care organization (MCO).
Methods:  The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Task Force on Good Research Practices—Use of Drug Costs for Cost Effectiveness Analysis (DCTF) was appointed by the ISPOR Board of Directors. Members were experienced developers or users of CEA models. The DCTF met to develop core assumptions and an outline before preparing a draft report. They solicited comments on drafts from external reviewers and from the ISPOR membership at ISPOR meetings and via the ISPOR Web site.
Results:  The cost of a drug to an MCO equals the amount it pays to the dispenser for the drug's ingredient cost and dispensing fee minus the patient copay and any rebates paid by the drug's manufacturer. The amount that an MCO reimburses for each of these components can differ substantially across a number of factors that include type of drug (single vs. multisource), dispensing site (retail vs. mail order), and site of administration (self-administered vs. physician's office). Accurately estimating the value of cost components is difficult because they are determined by proprietary and confidential contracts.
Conclusion:  Estimates of drug cost from the MCO perspective should include amounts paid for medication ingredients and dispensing fees, and net out copays, rebates, and other drug price reductions. Because of the evolving nature of drug pricing, ISPOR should publish a Web site where current DCTF costing recommendations are updated as new information becomes available.     

Good Research Practices for Measuring Drug Costs in Cost-Effectiveness Analyses: Medicare, Medicaid and Other US Government Payers Perspectives: The ISPOR Drug Cost Task Force Report—Part IV

Objectives:  Public programs finance a large share of the US pharmaceutical expenditures. To date, there are not guidelines for estimating the cost of drugs financed by US public programs. The objective of this study was to provide standards for estimating the cost of drugs financed by US public programs for utilization in pharmacoeconomic evaluations.
Methods:  This report was prepared by the ISPOR Task Force on Good Research Practices—Use of Drug Costs for Cost-Effectiveness Analysis Medicare, Medicaid, and other US Government Payers Subgroup. The Subgroup was convened to assess the methodological and practical issues confronted by researchers when estimating the cost of drugs financed by US public programs, and to propose standards for more transparent, accurate and consistent costing methods.
Results:  The Subgroup proposed these recommendations: 1) researchers must consider regulation requirements that affect the drug cost paid by public programs; 2) drug cost must represent the actual acquisition cost, incorporating any rebates or discounts; 3) transparency with respect to cost inputs must be ensured; 4) inclusion of the public program's perspective is recommended; 5) high cost drugs require special attention, particularly when drugs represent a significant proportion of health-care expenditures for a specific disease; and 6) because of variations across public programs, sensitivity analyses for actual acquisition cost, real-world adherence, and generics availability are warranted. Specific recommendations also were proposed for the Medicare and Medicaid programs.
Conclusions:  As pharmacoeconomic evaluations for coverage decisions made by US public programs grows, the need for precise and consistent estimation of drug costs is warranted. Application of the proposed recommendations will allow researchers to include accurate and unbiased cost estimates in pharmacoeconomic evaluations.     

On the aggregation of published prognostic scores for causal inference in observational studies

As real world evidence on drug efficacy involves nonrandomized studies, statistical methods adjusting for confounding are needed. In this context, prognostic score (PGS) analysis has recently been proposed as a method for causal inference. It aims to restore balance across the different treatment groups by identifying subjects with a similar prognosis for a given reference exposure (“control”). This requires the development of a multivariable prognostic model in the control arm of the study sample, which is then extrapolated to the different treatment arms. Unfortunately, large cohorts for developing prognostic models are not always available. Prognostic models are therefore subject to a dilemma between overfitting and parsimony; the latter being prone to a violation of the assumption of no unmeasured confounders when important covariates are ignored. Although it is possible to limit overfitting by using penalization strategies, an alternative approach is to adopt evidence synthesis. Aggregating previously published prognostic models may improve the generalizability of PGS, while taking account of a large set of covariates—even when limited individual participant data are available. In this article, we extend a method for prediction model aggregation to PGS analysis in nonrandomized studies. We conduct extensive simulations to assess the validity of model aggregation, compared with other methods of PGS analysis for estimating marginal treatment effects. We show that aggregating existing PGS into a “meta-score” is robust to misspecification, even when elementary scores wrongfully omit confounders or focus on different outcomes. We illustrate our methods in a setting of treatments for asthma.     

Application of Constrained Optimization Methods in Health Services Research: Report 2 of the ISPOR Optimization Methods Emerging Good Practices Task Force

Background

Constrained optimization methods are already widely used in health care to solve problems that represent traditional applications of operations research methods, such as choosing the optimal location for new facilities or making the most efficient use of operating room capacity.

Multinational Trials—Recommendations on the Translations Required, Approaches to Using the Same Language in Different Countries, and the Approaches to Support Pooling the Data: The ISPOR Patient-Reported Outcomes Translation and Linguistic Validation Good Research Practices Task Force Report

Objectives:  With the internationalization of clinical trial programs, there is an increased need to translate and culturally adapt patient-reported outcome (PRO) measures. Although guidelines for good practices in translation and linguistic validation are available, the ISPOR Patient-Reported Outcomes Translation and Linguistic Validation Task Force identified a number of areas where they felt that further discussion around methods and best practices would be beneficial.
The areas identified by the team were as follows: 1) the selection of the languages required for multinational trials; 2) the approaches suggested when the same language is required across two or more countries; and 3) the assessment of measurement equivalence to support the aggregation of data from different countries.
Methods:  The task force addressed these three areas, reviewed the available literature, and had multiple discussions to develop this report.
Results:  Decision aid tools have also been developed and presented for the selection of languages and the approaches suggested for the use of the same language in different countries.
Conclusion:  It is hoped that this report and the decision tools proposed will assist those involved with multinational trials to 1) decide on the translations required for each country; 2) choose the approach to use when the same language is spoken in more than one country; and 3) choose methods to gather evidence to support the pooling of data collected using different language versions of the same tool.     

Transferability of Economic Evaluations Across Jurisdictions: ISPOR Good Research Practices Task Force Report

A growing number of jurisdictions now request economic data in support of their decision-making procedures for the pricing and/or reimbursement of health technologies. Because more jurisdictions request economic data, the burden on study sponsors and researchers increases. There are many reasons why the cost-effectiveness of health technologies might vary from place to place. Therefore, this report of an ISPOR Good Practices Task Force reviews what national guidelines for economic evaluation say about transferability, discusses which elements of data could potentially vary from place to place, and recommends good research practices for dealing with aspects of transferability, including strategies based on the analysis of individual patient data and based on decision-analytic modeling.     

Good Research Practices for Measuring Drug Costs in Cost Effectiveness Analyses: Issues and Recommendations: The ISPOR Drug Cost Task Force Report—Part I

ObjectivesThe assignment of prices or costs to pharmaceuticals can be crucial to results and conclusions that are derived from pharmacoeconomic cost effectiveness analyses (CEAs). Although numerous pharmacoeconomic practice guidelines are available in the literature and have been promulgated in many countries, these guidelines are either vague or silent about how drug costs should be established or measured. This is particularly problematic in pharmacoeconomic studies performed from the “societal” perspective, because typically the measured cost of a brand name pharmaceutical is not a true economic cost but also includes transfer payments from some members of society (patients and third party payers) to other members of society (pharmaceutical manufacturer stockholders) in large part as a reward for biomedical innovation. Moreover, there are numerous and complex institutional factors that influence how drug costs should be measured from other CEA perspectives, both internationally and within the domestic US context. The objective of this report is to provide guidance and recommendations on how drug costs should be measured for CEAs performed from a number of key analytic perspectives.MethodsISPOR Task Force on Good Research Practices—Use of Drug Costs for Cost Effectiveness Analysis (Drug Cost Task Force [DCTF]) was appointed with the advice and consent of the ISPOR Board of Directors. Members were experienced developers or users of CEA models, worked in academia, industry, and as advisors to governments, and came from several countries. Because how drug costs should be measured for CEAs depend on the perspectives, five Task Force subgroups were created to develop drug cost standards from the societal, managed care, US government, industry, and international perspective. The ISPOR Task Force on Good Research Practices—Use of Drug Costs for Cost Effectiveness Analysis (DCTF) subgroups met to develop core assumptions and an outline before preparing six draft reports. They solicited comments on the outline and drafts from a core group of 174 external reviewers and more broadly from the membership of ISPOR at two ISPOR meetings and via the ISPOR web site.ResultsDrug cost measurements should be fully transparent and reflect the net payment most relevant to the user's perspective. The Task Force recommends that for CEAs of brand name drugs performed from a societal perspective, either 1) CEA analysts use a cost that more accurately reflects true societal drug costs (e.g., 20–60% of average sales price), or when that is too unrealistic to be meaningful for decision-makers, 2) refer to their analyses as from a “limited societal perspective.” CEAs performed from a payer perspective should use drug prices actually paid by the relevant payer net of all rebates, copays, or other adjustments. When such price adjustments are confidential, the analyst should apply a typical or average discount that preserves this confidentiality.ConclusionsDrug transaction prices not only ration current use of medication but also ration future biomedical research and development. CEA researchers should tailor the appropriate measure of drug costs to the analytic perspective, maintain clarity and transparency on drug cost measurement, and report the sensitivity of CEA results to reasonable drug cost measurement alternatives.     

Causal Inference in the Health Sciences: A Conceptual Introduction

This paper provides a conceptual introduction to causal inference, aimed to assist health services researchers benefit from recent advances in this area. The paper stresses the paradigmatic shifts that must be undertaken in moving from traditional statistical analysis to causal analysis of multivariate data. Special emphasis is placed on the assumptions that underlie all causal inferences, the languages used in formulating those assumptions, and the conditional nature of causal claims inferred from nonexperimental studies. These emphases are illustrated through a brief survey of recent results, including the control of confounding, corrections for noncompliance, and a symbiosis between counterfactual and graphical methods of analysis.     

A data-adaptive strategy for inverse weighted estimation of causal effects

In most nonrandomized observational studies, differences between treatment groups may arise not only due to the treatment but also because of the effect of confounders. Therefore, causal inference regarding the treatment effect is not as straightforward as in a randomized trial. To adjust for confounding due to measured covariates, the average treatment effect is often estimated by using propensity scores. Typically, propensity scores are estimated by logistic regression. More recent suggestions have been to employ nonparametric classification algorithms from machine learning. In this article, we propose a weighted estimator combining parametric and nonparametric models. Some theoretical results regarding consistency of the procedure are given. Simulation studies are used to assess the performance of the newly proposed methods relative to existing methods, and a data analysis example from the Surveillance, Epidemiology and End Results database is presented.     

Principles of Good Practice for Budget Impact Analysis: Report of the ISPOR Task Force on Good Research Practices—Budget Impact Analysis

OBJECTIVES: There is growing recognition that a comprehensive economic assessment of a new health-care intervention at the time of launch requires both a cost-effectiveness analysis (CEA) and a budget impact analysis (BIA). National regulatory agencies such as the National Institute for Health and Clinical Excellence in England and Wales and the Pharmaceutical Benefits Advisory Committee in Australia, as well as managed care organizations in the United States, now require that companies submit estimates of both the cost-effectiveness and the likely impact of the new health-care interventions on national, regional, or local health plan budgets. Although standard methods for performing and presenting the results of CEAs are well accepted, the same progress has not been made for BIAs. The objective of this report is to present guidance on methodologies for those undertaking such analyses or for those reviewing the results of such analyses. METHODS: The Task Force was appointed with the advice and consent of the Board of Directors of ISPOR. Members were experienced developers or users of budget impact models, worked in academia, industry, and as advisors to governments, and came from several countries in North America, Oceana, Asia, and Europe. The Task Force met to develop core assumptions and an outline before preparing a draft report. They solicited comments on the outline and two drafts from a core group of external reviewers and more broadly from the membership of ISPOR at two ISPOR meetings and via the ISPOR web site. RESULTS: The Task Force recommends that the budget impact of a new health technology should consider the perspective of the specific health-care decision-maker. As such, the BIA should be performed using data that reflect, for a specific health condition, the size and characteristics of the population, the current and new treatment mix, the efficacy and safety of the new and current treatments, and the resource use and costs for the treatments and symptoms as would apply to the population of interest. The Task Force recommends that budget impact analyses be generated as a series of scenario analyses in the same manner that sensitivity analyses would be provided for CEAs. In particular, the input values for the calculation and the specific cost outcomes presented (a scenario) should be specific to a particular decision-maker's population and information needs. Sensitivity analysis should also be in the form of alternative scenarios chosen from the perspective of the decision-maker. The primary data sources for estimating the budget impact should be published clinical trial estimates and comparator studies for efficacy and safety of current and new technologies as well as, where possible, the decision-maker's own population for the other parameter estimates. Suggested default data sources also are recommended. These include the use of published data, well-recognized local or national statistical information and in special circumstances, expert opinion. Finally, the Task Force recommends that the analyst use the simplest design that will generate credible and transparent estimates. If a health condition model is needed for the BIA, it should reflect health outcomes and their related costs in the total affected population for each year after the new intervention is introduced into clinical practice. The model should be consistent with that used for the CEA with regard to clinical and economic assumptions. CONCLUSIONS: The BIA is important, along with the CEA, as part of a comprehensive economic evaluation of a new health technology. We propose a framework for creating budget impact models, guidance about the acquisition and use of data to make budget projections and a common reporting format that will promote standardization and transparency. Adherence to these proposed good research practice principles would not necessarily supersede jurisdiction-specific budget impact guidelines, but may support and enhance local recommendations or serve as a starting point for payers wishing to promulgate methodology guidelines.     

Illustrating Emerging Good Practices for Quantitative Benefit-Risk Assessment: A Hypothetical Case Study of Systemic Biologic Treatments for Plaque Psoriasis

ObjectivesQuantitative benefit-risk assessment (qBRA) is a structured process to evaluate the benefit-risk balance of treatment options to support decision making. The ISPOR qBRA Task Force was recently established to provide recommendations for the design, conduct, and reporting of qBRA. This report presents a hypothetical case study illustrating how to apply the Task Force’s recommendations toward a qBRA to inform the benefit-risk assessment of brodalumab at the time of initial marketing approval. The qBRA evaluated 2 dosing regimens of brodalumab (210 mg or 140 mg twice weekly) compared with weight-based dosing of ustekinumab and placebo.MethodsWe followed the 5 steps recommended by the Task Force. Attributes included treatment response (≥75% improvement in Psoriasis Area and Severity Index), suicidal ideation and behavior, and infections. Performance data were drawn from pivotal clinical trials of brodalumab. The qBRA used multicriteria decision analysis and preference weights from a hypothetical discrete choice experiment. Sensitivity analyses examined the robustness of benefit-risk ranking to uncertainty in clinical effect and preference estimates, consideration of a subgroup (nail psoriasis), and the maintenance phase of treatment (52 weeks instead of 12).ResultsResults from this hypothetical qBRA suggest that brodalumab 210 mg had a more favorable benefit-risk profile compared with ustekinumab and placebo. Ranking of brodalumab compared with ustekinumab was dependent on brodalumab’s dose. Sensitivity analyses demonstrated robustness of benefit-risk ranking to uncertainty in clinical effect and preference estimates, as well as choice of attributes and length of follow-up.ConclusionThis case study demonstrates how to implement the ISPOR Task Force’s good practice recommendations on qBRA.     

Adjusting for confounded variables: pulmonary function and smoking in a special population

Confounded variables present an obstacle to valid inference in many environmental and occupational studies. We describe a series of procedures that we used to address this problem in a study of pulmonary function and smoking. Subjects were drawn from the Multiple Risk Factor Intervention Trial (MRFIT), a prospective study of coronary heart disease. Confounding of smoking, hypertension, and hyperlipidemia was designed into the trial and was beyond the control of our ancillary study. We used statistical techniques to detect and characterize the pattern of confounding, identify important variables affecting pulmonary function, and perform appropriate adjustments for extraneous influences (i.e., other than smoking). Among the techniques we used were factor analysis, stepwise multiple regression, and bootstrap replication. Analysis of the adjusted pulmonary function measurements showed that they were satisfactorily standardized and free of artifact. Moreover, use of the adjusted values sharpened our statistical results concerning smoking, the ultimate object of the study. We contrast the use of external and internal standards and discuss methods for detecting, ruling out, or counteracting confounding.