Acceptability and Perceived Benefits and Risks of Public and Patient Involvement in Health Care Policy: A Delphi Survey in Belgian Stakeholders

BackgroundIn systems with public health insurance, coverage decisions should reflect social values. Deliberation among stakeholders could achieve this goal, but rarely involves patients and citizens directly.ObjectivesThis study aimed at evaluating the acceptability, and the perceived benefits and risks, of public and patient involvement (PPI) in coverage decision making to Belgian stakeholders.MethodsA two-round Delphi survey was conducted among all stakeholder groups. The survey was constructed on the basis of interviews with 10 key stakeholders and a review of the literature on participation models. Consensus was defined as 65% or more of the respondents agreeing with a statement and less than 15% disagreeing. Eighty stakeholders participated in both rounds. They were defined as the Delphi panel.ResultsBelgian stakeholders are open toward PPI in coverage decision processes. Benefits are expected to exceed risks. The preferred model for involvement is to consult citizens or patients, within the existing decision-making structures and at specific milestones in the process. Consulting citizens and patients is a higher level of involvement than merely informing them and a lower level than letting them participate actively. Consultation involves asking nonbinding advice on (parts of) the decision problem. According to the Delphi panel, the benefits of PPI could be increasing awareness among members of the general public and patients about the challenges and costs of health care, and enriched decision processes with expertise by experience from patients. Potential risks include subjectivity, insufficient resources to participate and weigh on the process, difficulties in finding effective ways to express a collective opinion, the risk of manipulation, and lobbying or power games of other stakeholders.ConclusionsPPI in coverage decision-making processes is acceptable to Belgian stakeholders, be it in different ways for different types of decisions. Benefits are expected to outweigh risks.     

The efficiency frontier approach to economic evaluation of health‐care interventions

Background: IQWiG commissioned an international panel of experts to develop methods for the assessment of the relation of benefits to costs in the German statutory health‐care system. Proposed methods: The panel recommended that IQWiG inform German decision makers of the net costs and value of additional benefits of an intervention in the context of relevant other interventions in that indication. To facilitate guidance regarding maximum reimbursement, this information is presented in an efficiency plot with costs on the horizontal axis and value of benefits on the vertical. The efficiency frontier links the interventions that are not dominated and provides guidance. A technology that places on the frontier or to the left is reasonably efficient, while one falling to the right requires further justification for reimbursement at that price. This information does not automatically give the maximum reimbursement, as other considerations may be relevant. Given that the estimates are for a specific indication, they do not address priority setting across the health‐care system. Conclusion: This approach informs decision makers about efficiency of interventions, conforms to the mandate and is consistent with basic economic principles. Empirical testing of its feasibility and usefulness is required. Copyright © 2010 John Wiley & Sons, Ltd.     

Economic Evaluation of Vaccination Programs: A Guide for Selecting Modeling Approaches

ObjectivesIllustrate 3 economic evaluation methods whose value measures may be useful to decision makers considering vaccination programs.MethodsKeyword searches identified example publications of cost-effectiveness analysis (CEA), fiscal health modeling (FHM), and constrained optimization (CO) for economic evaluation of a vaccination program in countries where at least 2 of the methods had been used. We examined the extent to which different value measures may be useful for decision makers considering adoption of a new vaccination program. With these findings, we created a guide for selecting modeling approaches illustrating the decision-maker contexts and policy objectives for which each method may be useful.ResultsWe identified 8 countries with published evaluations for vaccination programs using >1 method for 4 infections: influenza, human papilloma virus, rotavirus, and malaria. CEA studies targeted health system decision makers using a threshold to determine the efficiency of a new vaccination program. FHM studies targeted public sector spending decision makers estimating lifetime changes in government tax revenue net of transfer payments. CO studies targeted decision makers selecting from a mix of options for preventing an infectious disease within budget and feasibility constraints. Cost and utility inputs, epidemiologic models, comparators, and constraints varied by modeling method.ConclusionsAlthough CEAs measures of incremental cost-effectiveness ratios are critical for understanding vaccination program efficiency for all decision makers determining access and reimbursement, FHMs provide measures of the program’s impact on public spending for government officials, and COs provide measures of the optimal mix of all prevention interventions for public health officials.     

Coûts de construction du programme de « promotion de la santé par les activités physiques et sportives et de prévention des conduites dopantes : PN-APSD – 2002–2007 »

BackgroundAccording to the framework legislation promulgated as part of the reform of finance laws in France, quality is a mandatory feature of all governmental actions. In this context, this work was conducted to assess the construction cost of a national health program designed to promote physical and sports activities and prevent doping behaviors. This program was considered to have the characteristic features of a successful governmental health intervention.MethodsFour cost categories were evaluated: cost of the activity itself, transportation costs, communication costs and promotion costs.ResultsIt was found that the program costs for 2002–2007 were 100,000 euros, with 15% of the costs in the communication category.ConclusionEconomic elements could be associated with factors of successful health service interventions in order to help decision makers responsible for the public interest and the consistency of public health actions.     

Exploring the use of economic evidence to inform investment in disease prevention – a qualitative study

Objective : In the context of growing financial pressures on health budgets, cost‐effective prevention strategies are needed to address the burden from non‐communicable disease in Australia. We explored how decision makers use economic evidence to inform such investment and how such evidence generated can more effectively meet the needs of end users. Methods : Thematic analysis of in‐depth interviews with 15 high level stakeholders (Treasury, state health departments and the insurance industry), supplemented by documentary analysis. Results : Types of prevention approaches and economic evidence relevant to decision makers differed by organisational perspective. Capacity building in understanding economic evaluations and research evidence that addresses the differing criteria for investment used by different organisations is needed. The task of determining investment priorities in disease prevention comes with significant challenges including ideological barriers, delayed outcome measures, and implementation uncertainties. Conclusions and Implications for public health : Promoting the greater use of economic evidence in prevention requires more work on two fronts: tailoring the methods used by economists to better match the organisational imperatives of end users; and promoting greater consideration of broader societal and health sector perspectives among end users. This will require significant infrastructure development, monitoring and evaluation, stronger national leadership and a greater emphasis on evidence coproduction.     

The influence of economic evaluation studies on decision making. A European survey. The EUROMET group

Despite the growing activity in the field of health economics very little is known about the influence of economic evaluation studies on health care decision making in the EU member states. Several investigations about the impact of health economic studies on decision making have been performed, but most of them did not involve decision makers themselves. In this paper the results of the EUROMET survey are reported and discussed. Different types of decision makers in nine European countries were surveyed by postal questionnaires, semi-structured interviews and focus group discussions. Questions include issues about the extent of knowledge about economic evaluation, the actual and potential use of study results as well as barriers and incentives in the use of studies. It is concluded that despite the general positive attitude knowledge about the formal methodology is rather limited. Accordingly, results of economic evaluation studies are not widely used in decision making. The results show that institutional dimensions, such as difficulties in transferring budgets, are viewed as important barriers. Also, the lack of credibility of studies is assigned a high relevance. Moreover, decision makers wish for a better explanation of the practical relevance of studies and feel that there is a need for more training in health economics. Considering these requirements a number of recommendations for enhancing the value of health economic studies are given.     

Can the impact of public involvement on research be evaluated? A mixed methods study

Background Public involvement is central to health and social research policies, yet few systematic evaluations of its impact have been carried out, raising questions about the feasibility of evaluating the impact of public involvement. Objective To investigate whether it is feasible to evaluate the impact of public involvement on health and social research. Methods Mixed methods including a two‐round Delphi study with pre‐specified 80% consensus criterion, with follow‐up interviews. UK and international panellists came from different settings, including universities, health and social care institutions and charitable organizations. They comprised researchers, members of the public, research managers, commissioners and policy makers, self‐selected as having knowledge and/or experience of public involvement in health and/or social research; 124 completed both rounds of the Delphi process. A purposive sample of 14 panellists was interviewed. Results Consensus was reached that it is feasible to evaluate the impact of public involvement on 5 of 16 impact issues: identifying and prioritizing research topics, disseminating research findings and on key stakeholders. Qualitative analysis revealed the complexities of evaluating a process that is subjective and socially constructed. While many panellists believed that it is morally right to involve the public in research, they also considered that it is appropriate to evaluate the impact of public involvement. Conclusions This study found consensus among panellists that it is feasible to evaluate the impact of public involvement on some research processes, outcomes and on key stakeholders. The value of public involvement and the importance of evaluating its impact were endorsed.     

Conceptual Framework for Standard Economic Evaluation of Physical Activity Programs in Primary Prevention

Economic evaluations of primary prevention physical activity programs have gained importance because of scarce resources in health-care-systems. A concept for economic evaluation should be based on the efficacy of physical activity, the standard methods of economic evaluation and the aims of public health. Previous publications have examined only parts of these components and have not developed a comprehensive conceptual framework; it is the objective of this article to develop such a framework. The derived method should aid decision makers and staff members of intervention programs in reviewing and conducting an economic evaluation. A literature search of articles was done using six electronic databases. Referenced works for standard methods and more comprehensive approaches for evaluation of preventive programs were studied. The newly developed conceptual framework for economic evaluation includes: (1) the type of physical activity program; (2) features of a selected study population; (3) the outcome dimension comprising exercise efficacy, reach, recruitment, response rate, maintenance, compliance and adverse health effects plus the social impact; and (4) the cost dimension consisting of program development costs, program implementation costs including the implementation, recruitment, program, participants?? time costs and savings resulting from the health effects of the intervention. Cost-effectiveness also depends on the methodology, such as the chosen perspective, data collection, valuation methods and discounting. If an intervention is not considered cost-effective, it is necessary to check each dimension to find possible failures in order to learn for future interventions. A more detailed economic evaluation is of utmost importance for improved comparability and transferability.     

The Delphi panel method for generating health information

The Delphi panel technique was tested as a component of a healthinformation system in 1991 in rural Kenya. Twenty-six panelmembers were selected from twelve villages in a sub-districtwith a population of 160000. Most of the panelists had poorreading and writing skills, so twelve interviewers were trainedto assist them in completing questionnaires on issues such ascommon illnesses in different age-groups and on priority interventions.Except for minor differences in the ranking of common illnesses,the Delphi study provided results consistent with a householdinterview survey simultaneously undertaken in the area. Theusefulness of Delphi panel studies in African health informationsystems and their costs when conducted by local staff are discussed.     

Strengthening capacity in developing countries for evidence-based public health: the data for decision-making project

Public health officials and the communities they serve need to: identify priority health problems; formulate effective health policies; respond to public health emergencies; select, implement, and evaluate cost-effective interventions to prevent and control disease and injury; and allocate human and financial resources. Despite agreement that rational, data-based decisions will lead to improved health outcomes, many public health decisions appear to be made intuitively or politically. During 1991-1996, the US Centers for Disease Control and Prevention implemented the US Agency for International Development funded Data for Decision-Making (DDM) Project. DDM goals were to: (a) strengthen the capacity of decision makers to identify data needs for solving problems and to interpret and use data appropriately for public health decisions; (b) enhance the capacity of technical advisors to provide valid, essential, and timely data to decision makers clearly and effectively; and (c) strengthen health information systems (HISs) to facilitate the collection, analysis, reporting, presentation, and use of data at local, district, regional, and national levels. Assessments were conducted to identify important health problems, problem-driven implementation plans with data-based solutions as objectives were developed, interdisciplinary, in-service training programs for mid-level policy makers, program managers, and technical advisors in applied epidemiology, management and leadership, communications, economic evaluation, and HISs were designed and implemented, national staff were trained in the refinement of HISs to improve access to essential data from multiple sources, and the effectiveness of the strategy was evaluated. This strategy was tested in Bolivia, Cameroon, Mexico, and the Philippines, where decentralization of health services led to a need to strengthen the capacity of policy makers and health officers at sub-national levels to use information more effectively. Results showed that the DDM strategy improved evidence-based public health. Subsequently, DDM concepts and practices have been institutionalized in participating countries and at CDC.     

Development and Validation of an Instrument to Measure Health-Related Out-of-Pocket Costs: The Cost for Patients Questionnaire

ObjectiveThe growth of healthcare spending is a major concern for insurers and governments but also for patients whose health problems may result in costs going beyond direct medical costs. To develop a comprehensive tool to measure direct and indirect costs of a health condition for patients and their families to various outpatient contexts.MethodsWe conducted a content and face validation including results of a systematic review to identify the items related to direct and indirect costs for patients or their families and an online Delphi to determine the cost items to retain. We conducted a pilot test-retest with 18 naive participants and analyzed data calculating intraclass correlation and kappa coefficients.ResultsAn initial list of 34 items was established from the systematic review. Each round of the Delphi panel incorporated feedback from the previous round until a strong consensus was achieved. After 4 rounds of the Delphi to reach consensus on items to be included and wording, the questionnaire had a total of 32 cost items. For the test-retest, kappa coefficients ranged from −0.11 to 1.00 (median = 0.86), and intraclass correlation ranged from −0.02 to 0.99 (median = 0.62).ConclusionsA rigorous process of content and face development was implemented for the Cost for Patients Questionnaire, and this study allowed to set a list of cost elements to be considered from the patient’s perspective. Additional research including a test-retest with a larger sample will be part of a subsequent validation strategy.     

Evaluation and decision making. The economist's point of view (author's transl)

This brief review summarizes six fundamental points concerning the practical application of evaluation results by decision makers. 1. Areas of agreement and disagreement between the viewpoints of doctors and economists; the necessity and the possibility of reconciling them by a systematic but flexible approach. 2. Basic factors which oblige us to seek this reconciliation between clinical responsibility to the patient and economic responsibility to society. 3. The concept of economic efficiency and the three types of economic analysis for achieving, it which are the most closely linked to epidemiology. 4. Some evidence of the lack of economic efficiency in the health services, and the slowness of improvements. 5. Five types of difficulties in the use of evaluation results by decision makers. 6. Some pragmatic propositions to improve the situation.     

The use of economic evaluation in health care: Australian decision makers' perceptions

Decision making about the alternative uses of health care resources is an issue of critical concern for governments and administrators in all health care systems. While many factors need to be taken into consideration when making these decisions, economic evaluation can help to determine the relative efficiency of different choices. Research in various countries suggests that economic evaluation is not being used by health care decision makers to the extent that health economists think that it should be. Interest in the use of economic evaluation is increasing in Australia but, to date, there has been no Australian research which looks at its use from the point of view of its potential users--the decision makers. This study fills that gap. It was found that there was a high level of awareness of economic evaluation among the group of decision makers interviewed and that some had used it in their decision making. However decisions often have to be made quickly and take into account factors other than efficiency, hence limiting the use of economics. Other problems limiting its use were availability of data and lack of expertise. Those interviewed suggested a number of ways in which the problems they identified could be overcome. In particular, they recommended that researchers doing economic evaluations should be more responsive to the needs of the decision makers using them.     

Discounting in decision making: the consistency argument revisited empirically

Discounting is one of the prominent topics of debate in health economics. While the standard practice in economic evaluation is to discount costs and effects alike with a 3-5% discount rate, many have raised questions about this practice. The debate sometimes seems trapped in Weinstein and Stason's consistency argument. In this paper, we use a set of health care programs--resembling Weinstein and Stason's hypothetical programs--to test whether appointed societal decision makers are consistent in their preferences over present and future costs and health effects, and whether they discount costs and effects at the same rate. Our results demonstrate these appointed decision makers to be fairly inconsistent on both issues, susceptible to the framing of problems and in part myopic. In other words, our respondents appear to be incapable of providing reasonable and consistent preferences between present and future costs, and health effects for use in economic evaluations. There is some support for the idea that rather than using constant and identical rates for costs and effects, real differences in health endowment over time (the growth rate for health) could serve as a basis for discount rates. Our respondents seem to relate their discount rate for health to their expectations about future life expectancy, but this also is dependent on the elicitation method.     

Outcome measures for clinical genetics services: a comparison of genetics healthcare professionals and patients' views

OBJECTIVES: To explore genetics professionals' and patients' views about which outcome domains are most appropriate to measure the patient benefits of using a clinical genetics service. METHODS: A postal Delphi survey was sent to: 115 consultant geneticists; 162 genetic counsellors; 156 support group representatives; 106 patients. The survey contained 19 outcome domains and respondents assessed the usefulness of each for clinical genetics services. RESULTS: The final professional panel comprised 115 genetics healthcare professionals and the patient panel comprised 72 patients. The outcome domains that achieved consensus (at least 75% of panel rated 'useful') for the patient and professional panels were: decision-making; knowledge of the genetic condition; perceived personal control; risk perception; satisfaction; meeting expectations; ability to cope; diagnosis accuracy; quality of life. Comparison of the ratings between the professional panel and the patient panel showed there was no statistical difference (chi(2), p<0.01) between the ratings ('useful' compared to 'not useful') for 14 of the 19 outcome domains but found differences for the perceived usefulness of: level of depression; health status; spiritual well-being; test accuracy; rate of termination. CONCLUSIONS: This Delphi survey identified nine outcome domains which are good starting points to develop a core set of outcome measures for evaluating clinical genetics services.     

Economic appraisal in the National Health Service: a survey.

Considerable emphasis is currently being placed on the pursuit of efficiency in health service provision. This paper reports the results of a survey of the use of economic appraisal to assist decision makers in choosing efficient courses of action. The survey group comprised National Health Service staff who had undertaken a correspondence course in health economics. The respondents were asked to identify issues arising locally where economic appraisal could have been applied but was not, and to suggest reasons why economic appraisal had not been used. They were then asked to give local examples of attempts to use economic appraisal and to indicate whether they were successful or what problems had been encountered. The results suggest that there is greater use of economic appraisal than is apparent from published sources but there is still not very much. The paper also summarises comments from the respondents on the decision making process in the National Health Service.     

What health care services does the public want and who should decide? Ask them!

Most of the parties involved in healthcare decisions – governments, politicians, healthcare professionals, pharmaceutical companies, special interest groups – actively work to make their desires known. In Israel the public is part of the decision committee; in Germany health care decision are made more or less without the public being involved. In a recently published IJHPR article, Giora Kaplan and Orna Baron-Epel raise the question of how well acquainted senior decision makers in the Israeli health system are with the public’s priorities regarding the services being considered for inclusion in the public funding list. This commentary speculates about the reasons for the discrepancies found in that article between the decision makers’ and the public’s view. Furthermore, it reports on survey results from Germany about who should be part of the decision making committee and briefly touches upon the situation in other OECD countries. While public opinion may not be the determining factor, all authors advocate a strengthening of the public’s contribution to the health care decision making process, including steps to make decision makers aware of public priorities on an ongoing basis.     

Factors of the innovation, organization, environment, and individual that predict the influence five systematic reviews had on public health decisions.

OBJECTIVE: To determine the extent to which systematic reviews of public health interventions influenced public health decisions and which factors were associated with influencing these decisions. METHODS: This cross-sectional follow-up survey evaluated the use of five systematic reviews in public health decision making. Independent variables included characteristics of the innovation, organization, environment, and individual. Primary data were collected using a telephone survey and a self-administered organizational demographics questionnaire. Public health decision makers in all 41 public health units in Ontario were invited to participate in the study. Multiple linear regression analyses on the five program decisions were conducted. RESULTS: The systematic reviews were perceived as having the greatest amount of influence on decisions related to program justification and program planning, and the least influence on program evaluation decisions. The greater the perception that one's organization valued the use of research evidence for decision making and that ongoing training in the critical appraisal of research literature was provided, the greater the perception of the influence the systematic review had on public health decisions. CONCLUSIONS: Organizational characteristics are important predictors of the use of systematic reviews in public health decision making. Future dissemination strategies need to promote the value of using systematic reviews for program decision making as well as promote ongoing training in critical appraisal among intended users in Ontario.     

Discounting future health benefits: the poverty of consistency arguments

In economic evaluation of health care, main stream practice is to discount benefits at the same rate as costs. But main papers in which this practice is advocated have missed a distinction between two quite different evaluation problems: (1) How much does the time of program occurrence matter for value and (2) how much do delays in health benefits from programs implemented at a given time matter? The papers have furthermore focused on logical and arithmetic arguments rather than on real value considerations. These ‘consistency arguments’ are at best trivial, at worst logically flawed. At the end of the day, there is a sensible argument for equal discounting of costs and benefits rooted in microeconomic theory of rational, utility maximising consumers' saving behaviour. But even this argument is problematic, first because the model is not clearly supported by empirical observations of individuals' time preferences for health, second because it relates only to evaluation in terms of overall individual utility. It does not provide grounds for claiming that decision makers with a wider societal perspective, which may include concerns for fair distribution, need to discount health benefits and costs equally. This applies even if health benefits are measured in monetary terms. Copyright © 2010 John Wiley & Sons, Ltd.