Value of Research and Value of Development in Early Assessments of New Medical Technologies

ObjectivesIn early stages of development of new medical technologies, there are conceptually separate but related societal decisions to be made concerning adoption, further development (i.e., technical improvement), and research (i.e., clinical trials) of new technologies. This article presents a framework to simultaneously support these three decisions from a societal perspective. The framework is applied to the 70-gene signature, a gene-expression profile for breast cancer, deciding which patients should receive adjuvant systemic therapy after surgery. The “original” signature performed on fresh frozen tissue (70G-FFT) could be further developed to a paraffin-based signature (70G-PAR) to reduce test failures.MethodsA Markov decision model comparing the “current” guideline Adjuvant Online (AO), 70G-FFT, and 70G-PAR was used to simulate 20-year costs and outcomes in a hypothetical cohort in The Netherlands. The 70G-PAR strategy was based on projected data from a comparable technology. Incremental net monetary benefits were calculated to support the adoption decision. Expected net benefit of development for the population and expected net benefit of sampling were calculated to support the development and research decision.ResultsThe 70G-PAR had the highest net monetary benefit, followed by the 70G-FFT. The population expected net benefit of development amounted to €91 million over 20 years (assuming €250 development costs per patient receiving the test). The expected net benefit of sampling amounted to €61 million for the optimal trial (n = 4000).ConclusionsWe presented a framework to simultaneously support adoption, development, and research decisions in early stages of medical technology development. In this case, the results indicate that there is value in both further development of 70G-FFT into 70G-PAR and further research.     

Cost-effectiveness as a price control

After a technology is developed, cost-effectiveness analysis can offer an economically sound approach to adoption decisions. Little attention has been paid, however, to the incentives these criteria induce for getting technologies to market in the first place. We argue that technology adoption procedures more fully take into account the key trade-off inherent in research and development: the decreased welfare of current patients as a result of higher prices versus the increased welfare of future patients as a result of the incentives for innovation that such prices provide. Empirical evidence from a case study of HIV/AIDS provides an illustration of our conclusions.     

The value of implementation and the value of information: combined and uneven development

AIM: In a budget-constrained health care system, the decision to invest in strategies to improve the implementation of cost-effective technologies must be made alongside decisions regarding investment in the technologies themselves and investment in further research. This article presents a single, unified framework that simultaneously addresses the problem of allocating funds between these separate but linked activities. METHODS: The framework presents a simple 4-state world where both information and implementation can be either at the current level or "perfect.' Through this framework, it is possible to determine the maximum return to further research and an upper bound on the value of adopting implementation strategies. The framework is illustrated through case studies of health care technologies selected from those previously considered by the UK National Institute for Health and Clinical Excellence (NICE). RESULTS: Through the case studies, several key factors that influence the expected values of perfect information and perfect implementation are identified. These factors include the maximum acceptable cost-effectiveness ratio, the level of uncertainty surrounding the adoption decision, the expected net benefits associated with the technologies, the current level of implementation, and the size of the eligible population. CONCLUSIONS: Previous methods for valuing implementation strategies have not distinguished the value of efficacy research and the value of strategies to change the level of implementation. This framework demonstrates that the value of information and the value of implementation can be examined separately but simultaneously in a single framework. This can usefully inform policy decisions about investment in health care services, further research, and adopting implementation strategies that are likely to differ between technologies.     

The option value of delay in health technology assessment.

Processes of health technology assessment (HTA) inform decisions under uncertainty about whether to invest in new technologies based on evidence of incremental effects, incremental cost, and incremental net benefit monetary (INMB). An option value to delaying such decisions to wait for further evidence is suggested in the usual case of interest, in which the prior distribution of INMB is positive but uncertain. METHODS: of estimating the option value of delaying decisions to invest have previously been developed when investments are irreversible with an uncertain payoff over time and information is assumed fixed. However, in HTA decision uncertainty relates to information (evidence) on the distribution of INMB. This article demonstrates that the option value of delaying decisions to allow collection of further evidence can be estimated as the expected value of sample of information (EVSI). For irreversible decisions, delay and trial (DT) is demonstrated to be preferred to adopt and no trial (AN) when the EVSI exceeds expected costs of information, including expected opportunity costs of not treating patients with the new therapy. For reversible decisions, adopt and trial (AT) becomes a potentially optimal strategy, but costs of reversal are shown to reduce the EVSI of this strategy due to both a lower probability of reversal being optimal and lower payoffs when reversal is optimal. Hence, decision makers are generally shown to face joint research and reimbursement decisions (AN, DT and AT), with the optimal choice dependent on costs of reversal as well as opportunity costs of delay and the distribution of prior INMB.     

Expected value of information and decision making in HTA

Decision makers within a jurisdiction facing evidence of positive but uncertain incremental net benefit of a new health care intervention have viable options where no further evidence is anticipated to:(1)adopt the new intervention without further evidence;(2)adopt the new intervention and undertake a trial; or(3)delay the decision and undertake a trial.Value of information methods have been shown previously to allow optimal design of clinical trials in comparing option (2) against option (1), by trading off the expected value and cost of sample information. However, this previous research has not considered the effect of cost of reversal on expected value of information in comparing these options. This paper demonstrates that, where a new intervention is adopted, the expected value of information is reduced under optimal decision making with costs of reversing decisions. Further, the paper shows that comparing expected net gain of optimally designed trials for option (2) vs (1) conditional on cost of reversal, and (3) vs (1) conditional on opportunity cost of delay allow systematic identification of an optimal decision strategy and trial design.     

A Conceptual Framework for Life-Cycle Health Technology Assessment

ObjectivesHealth technology assessment (HTA) uses evidence appraisal and synthesis with economic evaluation to inform adoption decisions. Standard HTA processes sometimes struggle to (1) support decisions that involve significant uncertainty and (2) encourage continued generation of and adaptation to new evidence. We propose the life-cycle (LC)-HTA framework, addressing these challenges by providing additional tools to decision makers and improving outcomes for all stakeholders.MethodsUnder the LC-HTA framework, HTA processes align to LC management. LC-HTA introduces changes in HTA methods to minimize analytic time while optimizing decision certainty. Where decision uncertainty exists, we recommend risk-based pricing and research-oriented managed access (ROMA). Contractual procurement agreements define the terms of reassessment and provide additional decision options to HTA agencies. LC-HTA extends value-of-information methods to inform ROMA agreements, leveraging routine, administrative data, and registries to reduce uncertainty.ResultsLC-HTA enables the adoption of high-value high-risk innovations while improving health system sustainability through risk-sharing and reducing uncertainty. Responsiveness to evolving evidence is improved through contractually embedded decision rules to simplify reassessment. ROMA allows conditional adoption to obtain additional information, with confidence that the net value of that adoption decision is positive.ConclusionsThe LC-HTA framework improves outcomes for patients, sponsors, and payers. Patients benefit through earlier access to new technologies. Payers increase the value of the technologies they invest in and gain mechanisms to review investments. Sponsors benefit through greater certainty in outcomes related to their investment, swifter access to markets, and greater opportunities to demonstrate value.     

A Questionnaire to Assess the Relevance and Credibility of Observational Studies to Inform Health Care Decision Making: An ISPOR-AMCP-NPC Good Practice Task Force Report

Evidence-based health care decisions are best informed by comparisons of all relevant interventions used to treat conditions in specific patient populations. Observational studies are being performed to help fill evidence gaps. Widespread adoption of evidence from observational studies, however, has been limited because of various factors, including the lack of consensus regarding accepted principles for their evaluation and interpretation. Two task forces were formed to develop questionnaires to assist decision makers in evaluating observational studies, with one Task Force addressing retrospective research and the other Task Force addressing prospective research. The intent was to promote a structured approach to reduce the potential for subjective interpretation of evidence and drive consistency in decision making. Separately developed questionnaires were combined into a single questionnaire consisting of 33 items. These were divided into two domains: relevance and credibility. Relevance addresses the extent to which findings, if accurate, apply to the setting of interest to the decision maker. Credibility addresses the extent to which the study findings accurately answer the study question. The questionnaire provides a guide for assessing the degree of confidence that should be placed from observational studies and promotes awareness of the subtleties involved in evaluating those.     

The impact of the UK NHS purchaser-provider split on the 'rational' introduction of new medical technologies

This article uses evidence from case studies of the introduction of three new medical technologies to explore the impact of the UK NHS purchaser--provider split on the diffusion of new medical technologies. A desirable policy objective is assumed to be the 'rational' diffusion of medical technologies according to evidence that they are clinically and cost effective. Theoretical mechanisms are identified through which diffusion could be controlled, and the case studies are used to explore the extent to which rational technology diffusion occurs in practice in the NHS. They illustrate the influence of purchasers and providers on the introduction and early use of new technologies and explore the extent to which research about clinical and cost effectiveness is used to inform decisions about technology adoption. The results demonstrate the limited influence of purchasers and the short term clinical and organisational objectives pursued by providers in relation to technology adoption. It is suggested that initiatives to promote rational technology diffusion might be most effective if they are focused on decision making in providers, and if they aim to balance the influence on decisions of administrative and financial information about the technologies with more systematic use of research about clinical and cost effectiveness.     

Die Nutzung von Informationswertanalysen in Entscheidungen über angewandte Forschung

Public decision makers face demands to invest in applied research in order to accelerate the adoption of new genetic tests. However, such an investment is profitable only if the results gained from further investigations have a significant impact on health care practice. An upper limit for the value of additional information aimed at improving the basis for reimbursement decisions is given by the expected value of perfect information (EVPI). This study illustrates the significance of the concept of EVPI on the basis of a probabilistic cost-effectiveness model of screening for hereditary hemochromatosis among German men. In the present example, population-based screening can barely be recommended at threshold values of 50,000 or 100,000?Euro per life year gained and also the value of additional research which might cause this decision to be overturned is small: At the mentioned threshold values, the EVPI in the German public health care system was ca. 500,000 and 2,200,000?Euro, respectively. An analysis of EVPI by individual parameters or groups of parameters shows that additional research about adherence to preventive phlebotomy could potentially provide the highest benefit. The potential value of further research also depends on methodological assumptions regarding the decision maker's time horizon as well as on scenarios with an impact on the number of affected patients and the cost-effectiveness of screening.     

Identifying key parameters in cost-effectiveness analysis using value of information: a comparison of methods

Decisions in health care must be made, despite uncertainty about benefits, risks, and costs. Value of information analysis is a theoretically sound method to estimate the expected value of future quantitative research pertaining to an uncertain decision. If the expected value of future research does not exceed the cost of research, additional research is not justified, and decisions should be based on current evidence, despite the uncertainty. To assess the importance of individual parameters relevant to a decision, different value of information methods have been suggested. The generally recommended method assumes that the expected value of perfect knowledge concerning a parameter is estimated as the reduction in expected opportunity loss. This method, however, results in biased expected values and incorrect importance ranking of parameters. The objective of this paper is to set out the correct methods to estimate the partial expected value of perfect information and to demonstrate why the generally recommended method is incorrect conceptually and mathematically.     

Value of Information Analytical Methods: Report 2 of the ISPOR Value of Information Analysis Emerging Good Practices Task Force

The allocation of healthcare resources among competing priorities requires an assessment of the expected costs and health effects of investing resources in the activities and of the opportunity cost of the expenditure. To date, much effort has been devoted to assessing the expected costs and health effects, but there remains an important need to also reflect the consequences of uncertainty in resource allocation decisions and the value of further research to reduce uncertainty. Decision making with uncertainty may turn out to be suboptimal, resulting in health loss. Consequently, there may be value in reducing uncertainty, through the collection of new evidence, to better inform resource decisions. This value can be quantified using value of information (VOI) analysis. This report from the ISPOR VOI Task Force describes methods for computing 4 VOI measures: the expected value of perfect information, expected value of partial perfect information (EVPPI), expected value of sample information (EVSI), and expected net benefit of sampling (ENBS). Several methods exist for computing EVPPI and EVSI, and this report provides guidance on selecting the most appropriate method based on the features of the decision problem. The report provides a number of recommendations for good practice when planning, undertaking, or reviewing VOI analyses. The software needed to compute VOI is discussed, and areas for future research are highlighted.     

HMOs, moral hazard and cost shifting in workers' compensation

Previous research has shown that workers respond to the economic incentives provided in workers' compensation. In particular, claim frequency rises with increased benefits, and claim duration, on net, seems to increase. Here we provide additional evidence of another incidence of behavioral responses to incentives. We find that doctors in health maintenance organizations (HMOs) have a greater tendency to classify claims as compensable under workers' compensation than do other physicians. Our evidence suggests that the rapid expansion of HMOs over the 1980-1990 period resulted in a significant increase in workers' compensation claim frequency.     

What constitutes evidence in hospital new drug decision making?

In National Health Service hospitals in the UK the introduction of new drugs is controlled by a local Drug and Therapeutics Committee (DTC), which is expected to apply the principles of evidence-based medicine (EBM). In the light of growing expenditure on drugs, there is interest in how the decisions are made that lead to the local acceptance or rejection of a new drug. In this study the DTCs of two general hospitals were observed, tape-recorded and analysed to determine what was considered as evidence and how it was used in decision making. Evidence, as constituted by DTC members, was issues that affected the decision-making process and included: clinical trial data, cost, pre-existing prescribing of the drug, pharmaceutical company activities, decisions of other DTCs, patient demand, clinician excitement, and personality of the applicant. Debate usually started with a discussion of the scientific evidence, then the cost would be considered. Often this evidence was either inadequate or insufficient enough for a locally implementable decision and further types of evidence would be brought in to try and estimate the likely impact of adopting the new drug. EBM, while used in decision making, was supplemented by local knowledge, although decisions were accounted for in the language of scientific rationality. Both abstract scientific rationality and the local rationality of practical healthcare provision were present in the decisions of the DTCs on the adoption, or otherwise, of new drugs into local formularies and healthcare. We suggest the coming together of local and abstract in local decision-making needs to be taken into account when formulating policy and providing decision support.     

Toward the 'tipping point': decision aids and informed patient choice

Preference-sensitive treatment decisions involve making value trade-offs between benefits and harms that should depend on informed patient choice. There is strong evidence that patient decision aids not only improve decision quality but also prevent the overuse of options that informed patients do not value. This paper discusses progress in implementing decision aids and the policy prospects for reaching a "tipping point" in the adoption of "informed patient choice" as a standard of practice.     

How Uncertainty Matters Under Risk Neutrality

It is typical in cost-effectiveness analysis to invoke a normative decision-making framework that assumes, as a starting point, that “a quality-adjusted life-year (QALY) is a QALY is a QALY.” The implication of this assumption is that the decision maker is risk neutral and that expected values could be considered sufficiently informative for a given “approve or reject” decision. Nevertheless, it seems intuitive that less uncertainty should be desirable and this has led some to incorporate “real” risk aversion (RA) into cost-effectiveness analysis.We illustrate in this article that RA is not always necessary to justify choosing more over less certain options. We show that for a risk neutral decision maker, greater uncertainty can make the approval of technology less likely in the presence of (1) model nonlinearities, (2) nonlinear opportunity costs, and (3) irreversible costs. We call these cases of “apparent” RA.Incorporating explicit risk preferences into decision making can be challenging; nevertheless, as we show here, it is not necessary to justify caring about uncertainty in approval decisions.     

Managing Healthcare Budgets in Times of Austerity: The Role of Program Budgeting and Marginal Analysis

Given limited resources, priority setting or choice making will remain a reality at all levels of publicly funded healthcare across countries for many years to come. The pressures may well be even more acute as the impact of the economic crisis of 2008 continues to play out but, even as economies begin to turn around, resources within healthcare will be limited, thus some form of rationing will be required. Over the last few decades, research on healthcare priority setting has focused on methods of implementation as well as on the development of approaches related to fairness and legitimacy and on more technical aspects of decision making including the use of multi-criteria decision analysis. Recently, research has led to better understanding of evaluating priority setting activity including defining ‘success’ and articulating key elements for high performance. This body of research, however, often goes untapped by those charged with making challenging decisions and as such, in line with prevailing public sector incentives, decisions are often reliant on historical allocation patterns and/or political negotiation. These archaic and ineffective approaches not only lead to poor decisions in terms of value for money but further do not reflect basic ethical conditions that can lead to fairness in the decision-making process. The purpose of this paper is to outline a comprehensive approach to priority setting and resource allocation that has been used in different contexts across countries. This will provide decision makers with a single point of access for a basic understanding of relevant tools when faced with having to make difficult decisions about what healthcare services to fund and what not to fund. The paper also addresses several key issues related to priority setting including how health technology assessments can be used, how performance can be improved at a practical level, and what ongoing resource management practice should look like. In terms of future research, one of the most important areas of priority setting that needs further attention is how best to engage public members.     

Cost-benefit studies of health care programs. Choosing methods for desired results.

This article summarizes the results of a critical review of several cost-benefit analyses (CBA) of health care programs. With pertinent examples, it is demonstrated that the results and conclusions of a study depend upon the assumptions and methods underlying the measurement of costs and benefits in a CBA. Given the incentives for an analyst to comply with desires of his sponsor, and given the scope of the choice available to an analyst among alternative assumptions and methods, it seems quite possible that desired results often dictate the assumptions and methods chosen. It is recommended that a policy maker should suspect an advocacy in the results and conclusions of every CBA. If CBAs are to be a true decision aid, a policy maker ought to obtain several of them, each of which assesses the costs and benefits of a given action plan using assumptions and methods substantially different from the other.     

Shared decision making in clinical medicine: past research and future directions

CONTENT: Shared medical decision making is a process by which patients and providers consider outcome probabilities and patient preferences and reach a health care decision based on mutual agreement. Shared decision making is best used for problems involving medical uncertainty. During the process the provider-patient dyad considers treatment options and consequences and explores the fit of expected benefits and consequences of treatment with patient preferences for various outcomes. This paper reviews the literature on shared medical decision making. Several questions are considered. Although several studies suggest that patients do not want to be involved in decision making, these studies typically fail to separate decisions about technical aspects of treatment from preferences for outcomes. There is considerable evidence that patients want to be consulted about the impact of treatment. Studies on the acceptability of shared decision making for physicians have produced inconsistent results. Shared decision making is more acceptable to younger and better-educated patients. It remains unclear whether shared decision making requires expensive video presentations or whether the same results can be obtained with simpler methods, such as the decision board. We conclude that shared medical decision making is an important development in health care. More research is necessary to identify the effects of shared decision making on patient satisfaction and health outcomes. Further, more research is necessary in order to evaluate the most effective methods for engaging patients in decisions about their own health care.     

Assessing the Impact of Censoring of Costs and Effects on Health-Care Decision-Making: an Example Using the Atrial Fibrillation Follow-up Investigation of Rhythm Management (AFFIRM) Study

Objectives:  Losses to follow-up and administrative censoring can cloud the interpretation of trial-based economic evaluations. A number of investigators have examined the impact of different levels of adjustment for censoring, including nonadjustment, adjustment of effects only, and adjustment for both costs and effects. Nevertheless, there is a lack of research on the impact of censoring on decision-making. The objective of this study was to estimate the impact of adjustment for censoring on the interpretation of cost-effectiveness results and expected value of perfect information (EVPI), using a trial-based analysis that compared rate- and rhythm-control treatments for persons with atrial fibrillation.
Methods:  Three different levels of adjustment for censoring were examined: no censoring of cost and effects, censoring of effects only, and censoring of both costs and effects. In each case, bootstrapping was used to estimate the uncertainty incosts and effects, and the EVPI was calculated to determine the potential worth of further research.
Results:  Censoring did not impact the adoption decision. Nevertheless, this was not the case for the decision uncertainty or the EVPI. For a threshold of $50,000 per life-year, the EVPI varied between $626,000 (partial censoring) to $117 million (full censoring) for the eligible US population.
Conclusions:  The level of adjustment for censoring in trial-based cost-effectiveness analyses can impact on the decisions to fund a new technology and to devote resources for further research. Only when censoring is taken into account for both costs and effects are these decisions appropriately addressed.