The influence of technology assessments by advisory bodies on health policy and practice

Assessments of 45 health technologies undertaken by Australian advisory bodies are considered in terms of the effect they have had on health care policy. The conduct and impact of the assessments was influenced by selection of topics, timing, other inputs to policy formulation and changes to policy areas and assessment agencies. Seventeen of 26 detailed assessments appear to have influenced policy, with greatest impact occurring when the technology was being introduced and there was collection of local primary data. Continued requests for the assessment reports suggests a possible longer term educational effect. All nine appraisals of proposals for nationally funded centres were influential, but the influence of guidelines for superspecialty services is less clear. Factors decreasing the influence of assessments included limitations of available policy instruments, lack of consensus between governments and professional groups and pressure of more urgent matters on the policy process.     

Who Does the Numbers? The Role of Third-Party Technology Assessment to Inform Health Systems' Decision-Making about the Funding of Health Technologies

Background:  There is an increasing number of health-care systems using economic evaluations to inform decisions about the reimbursement of health technologies. There are usually two separate elements of this process: assembling relevant evidence and undertaking analyses (technology assessment), and decision-making. In most systems, technology assessment is undertaken by the manufacturer of the technology. In a few, "third-party" assessment is used.
Methods:  In the United Kingdom, the National Institute for Health and Clinical Excellence used a combination of third-party and manufacturer assessments between 1999 and 2005. After this point, a Single Technology Appraisal program (using manufacturer-based assessment) was instituted for some technologies. Here the role of third-party assessment is considered in this from of decision-making. The article reviews the requirements of economic evaluation to support decision-making, and considers the extent to which each type of assessment is likely to meet these requirements. It also attempts to address whether the two forms of assessment differ in their impact on decision-making using a comparison of the decisions made by National Institute for Health and Clinical Excellence (NICE) (under its multiple-technology appraisal system) and the Scottish Medicines Consortium (SMC), which relies on manufacturer assessment.
Results:  The comparison is limited by the small number of technologies considered by both bodies. Nevertheless, it suggests that there are potentially important differences between the two bodies, with NICE generally placing more restrictions of the use of technologies.
Conclusions:  The article concludes that there are potential advantages to third-party assessment, but its cost and timing may preclude its use for all new technologies. A hybrid arrangement is suggested where third-party assessment is used in particular circumstances.     

Evaluation of the impact of a technology appraisal process in England: the South and West Development and Evaluation Committee

OBJECTIVES: To examine the influence of the English National Health Service (NHS) South and West (S&W) region Development and Evaluation Committee (DEC) technology appraisal reports on purchasing and clinical decisions in the S&W and elsewhere in the NHS, and to estimate the cost of the DEC process. METHODS: Two postal questionnaires examining issues of awareness, influence and quality of the DEC reports were sent to samples of clinical and managerial health service staff in S&W NHS region, the North West NHS region, and three other randomly selected health authorities elsewhere. Supplementary telephone interviews were undertaken with a subsample of staff. Trends in utilisation before and after the publication of four S&W DEC reports were examined using routine health service data. RESULTS: Survey response rates ranged between 73% and 85%. Within health authorities in the S&W, agreement about the impact of the DEC in informing policy and purchasing decisions was consistently high, although responses from clinicians were more equivocal. In the topic areas examined, the impact of the DEC reports could not be separated from wider influences on practice. The cost of each DEC report is estimated to have been pounds sterling 24659. CONCLUSIONS: Health service staff perceived that the S&W DEC process had had an impact on policy decisions and clinical practice locally; however, any actual impact on practice could not be identified with routine data. The approach used in this study would be valuable in evaluating the impact of the English NHS National Institute for Clinical Excellence.     

Factors of the innovation, organization, environment, and individual that predict the influence five systematic reviews had on public health decisions.

OBJECTIVE: To determine the extent to which systematic reviews of public health interventions influenced public health decisions and which factors were associated with influencing these decisions. METHODS: This cross-sectional follow-up survey evaluated the use of five systematic reviews in public health decision making. Independent variables included characteristics of the innovation, organization, environment, and individual. Primary data were collected using a telephone survey and a self-administered organizational demographics questionnaire. Public health decision makers in all 41 public health units in Ontario were invited to participate in the study. Multiple linear regression analyses on the five program decisions were conducted. RESULTS: The systematic reviews were perceived as having the greatest amount of influence on decisions related to program justification and program planning, and the least influence on program evaluation decisions. The greater the perception that one's organization valued the use of research evidence for decision making and that ongoing training in the critical appraisal of research literature was provided, the greater the perception of the influence the systematic review had on public health decisions. CONCLUSIONS: Organizational characteristics are important predictors of the use of systematic reviews in public health decision making. Future dissemination strategies need to promote the value of using systematic reviews for program decision making as well as promote ongoing training in critical appraisal among intended users in Ontario.     

Linking evidence from health technology assessments to policy and decision making: the Alberta model

OBJECTIVES: The objective of this study was to develop and implement a decision-making process for public funding of health services that links policy areas with health technology assessment and input from interested parties. METHODS: Health authorities, assessment organizations, and healthcare professionals were consulted as a follow-up to recommendations of an expert panel established by the Alberta government. The methods involved formulation of an eight-stage, collaborative process that incorporates identification through the health ministry of health technologies requiring review, assessment of the technologies using expert groups in Alberta, and consultation and formulation of advice within the ministry to inform ministerial funding decisions. RESULTS: All components of the decision process have been put in place and have collaborated to provide advice to inform policy on provincial health services. Of nineteen technologies selected for review, five have been completed and decisions made: laparoscopic adjustable gastric banding, fetal fibronectin assay for premature labor, newborn screening for cystic fibrosis, newborn screening for inborn errors of metabolism, and gastric electrical stimulation. A further six reviews are in progress, and reviews of the remaining technologies are planned for 2007. CONCLUSIONS: Bridging the evidence-to-policy gap is more likely to succeed when the policy community is actively engaged and an explicit model is used to put health technology assessment into practice.     

Elements for assessment of telemedicine applications.

OBJECTIVES: As an initiative of the International Network of Agencies for Health Technology Assessment, an approach to assessment of telemedicine applications was prepared to assist decision makers who are considering introduction and use of this technology. METHODS: Review and commentary drawing on published assessment frameworks and reports of primary evaluations of telemedicine, with particular reference to experience in Finland and Canada. RESULTS: Elements of the approach included development of a business case (considering population and services, personnel and consumers, delivery arrangements, specifications and costs); subsequent evaluation of the telemedicine application; and follow-up (covering the domains of technical assessment, effectiveness, user assessment of the technology, costs of telemedicine, trials, economic evaluation methods, and sensitivity analysis). CONCLUSIONS: Decision makers should link introduction of new and often costly technology to appraisal of its feasibility, followed by evaluation of the application, including longer term consideration of its sustainability and impact on the healthcare system. As the effectiveness and efficiency of telemedicine applications will often be strongly influenced by local issues, results of assessments may not be generalizable.     

Impact of health technology assessment on preventive screening in Belgium: case studies of mammography in breast cancer, PSA screening in prostate cancer, and ultrasound in normal pregnancy

OBJECTIVE: To describe how scientific evidence has influenced healthcare policy making in Belgium in the field of sickness prevention for mammography, PSA testing in prostate cancer screening, and use of ultrasound in pregnancy. METHODS: Review of published and gray literature and interviews with stakeholders and experts. RESULTS: At the end of 1999, a systematic national/regional screening program had not yet been implemented for any of the three screening strategies. A systematic breast cancer screening program is being prepared for implementation only in Flanders. This limited impact can be attributed to the fragmentation in healthcare policy, the different options among the different regions, fragmentation in healthcare practice, the strong impact of healthcare stakeholders (provider groups and sickness funds) on decision making, and limited attention to scientific evidence in health policy and technology assessment. CONCLUSIONS: Health technology assessment has had very little impact on policy and practice in use of mammography, PSA testing, and ultrasound in pregnancy in Belgium.     

Extracorporeal shock wave therapy in orthopedics. Assessment of an emerging health technology

OBJECTIVE: Extracorporeal shock wave therapy (ESWT) as an emerging technology in orthopedics has been assessed in Austria with the objective to establish a scientific basis for pending and pressing health policy decisions. Despite encouraging results within some indications and the promising prospect of a noninvasive treatment for some orthopedic diseases, it seemed crucial to assess this new field of application in the light of evidence-based standards, without forgetting the reality of healthcare decisions. This article presents the results of the above mentioned assessment in the context of the policy-making process in Germany, Austria, and Switzerland. METHODS: The study is based on an overview of the already existing reviews on the effectiveness of ESWT as carried out by different actors (health insurances and orthopedists), and on an overview of additional literature covering all relevant aspects (mode of action, recommendations, economic estimates). Orthopedic societies, several institutions involved in health technology assessment, and health insurance agencies were contacted in search of further relevant literature, ongoing studies, and assessments. RESULTS: Until now, clear evidence has not been available on the effectiveness of ESWT in four currently recommended indications (tendinosis calcarea of the shoulder, epicondylopathia humeri radialis, calcaneal spur, and pseudarthrosis), although a substantial body of literature has been produced. Despite encouraging results in some indications, the lack of coherent therapy schemes and the poor quality in the design of clinical studies has been serious enough to hinder the drawing of reasonable conclusions. CONCLUSIONS: Due to the rapid diffusion of ESWT and the pressure on reimbursement agencies to prove the ineffectiveness of ESWT, the policy option of a pragmatic approach is recommended to control the diffusion of an uncertain health technology, alongside active research that would allow a more comprehensive appraisal of this potentially interesting treatment strategy in orthopedics.     

Gesundheitsökonomie und HTA

For a comprehensive health technology assessment (HTA), health economic studies are analysed as well as medical studies. Significance and effects of HTA are rather different in European countries. In Germany HTA is conducted by two official agencies: DIMDI and IQWiG. While DIMDI requires health economic assessments, IQWiG is prohibited from considering economic aspects by law (pure assessment of benefits). For the health economic assessment in HTA reports mainly secondary scientific evidence is exploited, i.e. primarily cost effectiveness and cost utility analyses. The study perspective, the study alternatives and the modelling methods are of great importance for the evaluation and have to be justified in depth. The cost effectiveness ratio and the budget impact are the most important results of the health economic assessment, which is the basis for the appraisal by the health care authorities. For this decision medical, utilitarian or other aspects like the avoidance of unwanted distribution effects are considered. On the other hand often only a few studies are available for the decision making when the appraisal has to be done. This problem might be less relevant in the near future as the number of HTAs and the number of health economic evaluations increases.     

The influence of health needs assessment on health care decision-making in London health authorities.

OBJECTIVES: Health needs assessment gained prominence under the model of health care purchasing developed to support the 1991 reforms of the UK National Health Service (NHS). The objectives of this paper are to determine how needs assessment has been used in the NHS, to assess the influence it has had on decision-making, and to relate the observed uses of needs assessment to competing theoretical models of health care policy-making. METHODS: A survey of needs assessment activity in 14 London health authorities identified 217 needs assessments conducted between 1993 and 1996. Semi-structured interviews were conducted with public health and commissioning staff in each authority. RESULTS: The survey indicated that needs assessment directly supported decision-making and action in two-thirds of the studies identified, but up to 20% of needs assessments had no impact on service provision. Four key functions of health needs assessment were observed: identifying a problem; planning detailed changes to services; providing post hoc justification for earlier decisions; and using participation in needs assessment to build 'ownership' of subsequent decisions. CONCLUSIONS: The survey suggests that needs assessment is, in practice, consistent with a 'mixed scanning' model of decision-making. Needs assessment is used to help select issues for detailed investigation and to direct analytical and decision-making resources. However, certain key areas are not amenable to technical analysis and solution, and are resolved through bargaining.     

Health technology assessment in Denmark

The Danish healthcare services are mainly provided by public sector institutions. The system is highly decentralized. The state has little direct influence on the provision of healthcare services. State influence is exercised through legislation and budget allocations. The main task of the state is to initiate, coordinate, and advise. Counties, which run the hospitals, also decide on the placement of services. The hospital sector is controlled within the framework of legislation and global budgets. General practitioners occupy a central position in the Danish healthcare sector, acting as gatekeepers to the rest of the system. The system works well, and its structure has resulted in steady costs of health care for a long period. There is no regulatory mechanism in the Danish health services requiring use of health technology assessment (HTA) as a basis for policy decisions, planning, or administrative procedures. However, since the late 1970s a number of comprehensive assessments of health technology have formed the basis for national health policy decisions. In 1997, after years of public criticism of the quality of hospital care and health technologies, and on the basis of a previously developed national HTA strategy, a national institute for HTA (DIHTA) was established. There seems to be a growing awareness of evidence-based healthcare among health professionals and a general acceptance of health economic analyses as a basis for health policy decision making. This progress is coupled with growing regional HTA activity in the health services. HTA seems to have a bright future in Denmark.     

Assessing the health benefits of advice services: using research evidence and logic model methods to explore complex pathways

Poverty is positively associated with poor health; thus, some healthcare commissioners in the UK have pioneered the introduction of advice services in health service locations. Previous systematic reviews have found little direct evidence for a causal relationship between the provision of advice and physical health and limited evidence for mental health improvement. This paper reports a study using a broader range of types of research evidence to construct a conceptual (logic) model of the wider evidence underpinning potential (rather than only proven) causal pathways between the provision of advice services and improvements in health. Data and discussion from 87 documents were used to construct a model describing interventions, primary outcomes, secondary and tertiary outcomes following advice interventions. The model portrays complex causal pathways between the intervention and various health outcomes; it also indicates the level of evidence for each pathway. It can be used to inform the development of research designed to evaluate the pathways between interventions and health outcomes, which will determine the impact on health outcomes and may explain inconsistencies in previous research findings. It may also be useful to commissioners and practitioners in making decisions regarding development and commissioning of advice services.     

Informing rural and remote students about careers in health: factors influencing career decisions

OBJECTIVE: Since 1994 a state-wide program has been operating in Queensland to provide non-metropolitan secondary school students with information about health careers. Determination of the factors influencing the career decisions of rural and remote students was one objective of the evaluation of that program. DESIGN: Telephone interviews. SETTING: Secondary schools. SUBJECTS: People who had previously attended Year 10 residential health career workshops run by Queensland Health. Ninety-four of 277 past participants to Year 10 Health Careers Workshops responded to a postal questionnaire and were invited to take part in a telephone questionnaire to collect further information related to the course and career choices. MAIN OUTCOME MEASURES: Self-reported factors affecting career decisions of students from rural and remote areas. RESULTS: Of the 94 past workshop participants, interviews were held with 70. All participants had left secondary education and had either undertaken or were undertaking some form of tertiary study. The most influential of nine factors to both undergraduate and postgraduate course choices was self-interest. Various other factors differentially contributed to undergraduate and postgraduate course choices among them were the health careers workshops, which played a major influence on undergraduate course decisions of students. CONCLUSION: Self-interest is the largest self-reported factor to career decision making among students from rural and remote areas. Finance and career advice from school are not considered to be highly influential. However, these independent decisions can be substantially influenced by external information provided through health career workshops.     

Protecting health using an environmental impact assessment: a case study of San Francisco land use decisionmaking

Laws and regulations for an environmental impact assessment enable a health impact assessment whenever physical changes in the environment may significantly affect health. In this case study, I describe 2 instances in which a local public health agency used the procedural requirements for an environmental impact assessment to account for societal-level health determinants that are not traditionally evaluated in land-use decisions. These examples show that a public health critique can contribute both to the scope of analysis in an environmental impact assessment and to substantive changes in land-use decisions. I have evaluated this health appraisal approach as a form of a health impact assessment and will make recommendations for law, research, and practice that support its technical, cultural, and political feasibility.     

How to design the cost‐effectiveness appraisal process of new healthcare technologies to maximise population health: A conceptual framework

This paper presents a conceptual framework to analyse the design of the cost‐effectiveness appraisal process of new healthcare technologies. The framework characterises the appraisal processes as a diagnostic test aimed at identifying cost‐effective (true positive) and non‐cost‐effective (true negative) technologies. Using the framework, factors that influence the value of operating an appraisal process, in terms of net gain to population health, are identified. The framework is used to gain insight into current policy questions including (a) how rigorous the process should be, (b) who should have the burden of proof, and (c) how optimal design changes when allowing for appeals, price reductions, resubmissions, and re‐evaluations. The paper demonstrates that there is no one optimal appraisal process and the process should be adapted over time and to the specific technology under assessment. Optimal design depends on country‐specific features of (future) technologies, for example, effect, price, and size of the patient population, which might explain the difference in appraisal processes across countries. It is shown that burden of proof should be placed on the producers and that the impact of price reductions and patient access schemes on the producer's price setting should be considered when designing the appraisal process.     

Assessing the Economic Impact of Telemedicine

Telemedicine is a communications technology that can be expected to influence investment in equipment, the delivery and organisation of health services and outcomes of healthcare. Analysis of the economic impact of telemedicine has so far been limited and has focused on cost studies for individual applications. Telemedicine initiatives should be compared with conventional alternatives. A search of recent literature revealed 20 such studies that included, usually limited, economic analyses. It is important to note, however, that such assessments apply to localised situations and may not be generalisable. Overall, the potential impact of telemedicine on healthcare systems remains unclear.The challenges that lie ahead for telemedicine, and its economic assessment, include technological change, sustainability of applications, availability of outcomes and other data, and generalisability of evaluation results.     

Rapid assessments"--quick evaluation of medical technology

Assessing the clinical, economical and other consequences of medical technologies (health technology assessment, HTA) is an important instrument to support decisions in many health care systems. A comprehensive HTA, however, is time-consuming and costly while, on the other hand, decisions have to be made quickly. A number of international HTA programmes established rapid assessment tracks although they are variable in scope and methods. In the first part of this paper, these programmes were compared with respect to scope, methods and time to complete assessments. It can be shown from this comparison that there is no common definition of "rapid assessments". In the second part of the paper, a model for processing rapid assessment in the German context is introduced and discussed. The model aims at rapid assessments serving the needs of German decision-makers and ensuring high scientific quality at the same time. The model consists of a modular system that is tailored to the actual demand of the decision-maker. Modules are obligatory (such as a systematic literature search) or optional (such as meta-analysis). All modules are subject to standardisation as far as possible. However, it should be kept in mind that a close collaboration between commissioners and executives of an HTA is necessary to focus on the question and work out the HTA accordingly.     

Health technology assessment for the NHS in England and Wales

In healthcare decision making, there is an important functional separation between assessment and appraisal. In the U.K. National Health Service (NHS), this distinction is illustrated by the separation of roles between the Health Technology Assessment Programme and the National Institute for Clinical Excellence. However, it can be seen at every level within the healthcare system. Assessment of a technology is a scientific task that synthesizes all relevant evidence on effectiveness and cost-effectiveness; its results are therefore generalizable. Appraisal of the technology is informed by the assessment but adds context-specific judgments on the applicability of the evidence, the feasibility and impact of alternative options, relative priorities, and wider social and ethical aspects. An explicit distinction between assessment and appraisal is helpful in achieving clarity, consistency, and consensus. It also makes clear the need for a wide range of assessment reviews to support decision making by commissioners, providers, and users of health services. Increasingly, the secondary research supported by the NHS R&D Programme is being distributed electronically. It is also being used to identify areas in which further primary research should be commissioned.     

Mini-symposium — Public Health Observatories: Supporting health impact assessment in practice

Health impact assessment (HIA) is a process that aims to predict potential positive and negative effects of project, programme or policy proposals on health and health inequalities. It is recommended by national government and internationally. Supporting health impact assessment is one of the roles of English Public Health Observatories. The few centres in England with accredited health impact training centres have inadequate resources to meet demand. Currently, the London Health Observatory is providing the bulk of the training nationally. Some Public Health Observatories are currently investigating the preferences for support of those commissioning or conducting health impact assessment within their regions. The availability of published guidance on how to conduct health impact assessments has increased substantially over the past few years. The Department of Health has funded a research project led by the London Health Observatory to develop advice for reviewing evidence for use in health impact assessment. Completed health impact assessments can be useful resources. Evaluation of the process and impact of health impact assessment is important in order to demonstrate its usefulness and to learn lessons for the future. The focus for Public Health Observatories is to train and support others to conduct health impact assessment according to good practice, rather than undertaking health impact assessments themselves. The aim is to create sufficient skilled capacity around the country to undertake health impact assessments. The London Health Observatory plans to share its support models and to roll out a train the trainer programme nationally to enable effective local delivery of their national health impact assessment programme.     

Supporting health impact assessment in practice

Health impact assessment (HIA) is a process that aims to predict potential positive and negative effects of project, programme or policy proposals on health and health inequalities. It is recommended by national government and internationally. Supporting health impact assessment is one of the roles of English Public Health Observatories. The few centres in England with accredited health impact training centres have inadequate resources to meet demand. Currently, the London Health Observatory is providing the bulk of the training nationally. Some Public Health Observatories are currently investigating the preferences for support of those commissioning or conducting health impact assessment within their regions. The availability of published guidance on how to conduct health impact assessments has increased substantially over the past few years. The Department of Health has funded a research project led by the London Health Observatory to develop advice for reviewing evidence for use in health impact assessment. Completed health impact assessments can be useful resources. Evaluation of the process and impact of health impact assessment is important in order to demonstrate its usefulness and to learn lessons for the future. The focus for Public Health Observatories is to train and support others to conduct health impact assessment according to good practice, rather than undertaking health impact assessments themselves. The aim is to create sufficient skilled capacity around the country to undertake health impact assessments. The London Health Observatory plans to share its support models and to roll out a train the trainer programme nationally to enable effective local delivery of their national health impact assessment programme.