Development and Pilot Test of the Registry Evaluation and Quality Standards Tool: An Information Technology–Based Tool to Support and Review Registries

ObjectivesHealth technology assessment (HTA) bodies are increasingly making use of real-world evidence and data. High-quality registries could be an asset for this; nevertheless, there is a lack of specified standards to assess the quality of data in the registry, or the registry itself. The European Network for Health Technology Assessment Joint Action 3 led the work to develop a tool for the evaluation of clinical registries: the “Registry Evaluation and Quality Standards Tool” (REQueST).MethodsREQueST was developed in 4 steps: (1) A partnership between HTA bodies across Europe drafted the assessment criteria. (2) Multiple rounds of consultation across HTA bodies and the public domain developed an Excel version of REQueST. (3) This version was transformed into a web-based application. (4) An external pilot tested this REQueST tool with SMArtCARE and NeuroTransData registries.ResultsHaute Autorité de Santé, the National Institute for Health and Care Excellence, and the Croatian Institute of Public Health led the development of REQueST. Another 4 HTA bodies contributed regularly to development meetings, and all European Network for Health Technology Assessment partners were invited to contribute. Eight methodological, 12 essential, and 3 supplementary criteria were identified. Both pilot registries scored well, fulfilling the requirements for >70% of criteria, with none failed. Feedback by registry holders led to streamlining of the process and clarification of the criteria.ConclusionsThe REQueST tool uses an iterative and collaborative methodology with registry holders. It has the potential to maximize the utility of registry data for decision making by regulatory and HTA bodies and provides a foundation for future research.     

Evaluation of the National Institute for Health and Care Excellence Diagnostics Assessment Program Decisions: Incremental Cost-Effectiveness Ratio Thresholds and Decision-Modifying Factors

ObjectivesThe National Institute for Health and Care Excellence (NICE) Diagnostics Assessment Programme (DAP) evaluates the cost-effectiveness of diagnostic technologies. A decision-making process benchmarking the incremental cost-effectiveness ratio (ICER) against a threshold while considering decision-modifying factors is common to NICE evaluations. This study investigated whether DAP decisions are consistent with the ICER thresholds described in the DAP manual, and to assess the impact of decision-modifying factors.MethodsDAP evaluations published before March 2018 were reviewed, and the following items were extracted: diagnostic technologies evaluated, decision problems assessed, Diagnostics Advisory Committee (DAC) decisions, incremental quality-adjusted life years (QALYs), incremental costs, ICERs considered to be most plausible by the DAC, and decision justifications.ResultsAll 30 evaluations were reviewed; 8 were excluded because the DAC concluded there was “insufficient evidence” for decision making. In the remaining 22 evaluations, 91 decision problems were identified for further analysis, of which 52, 15, and 24 received “recommended,” “not recommended,” and “not recommended–only in research” guidance, respectively. The overall consistency rate of the DAC decisions with the £20 000/QALY threshold was 73.6%. Diagnostic technologies that were not recommended, despite an ICER less than £20 000/QALY, were associated with a larger number of decision-modifying factors favoring the comparator, versus recommended diagnostic technologies with ICERs less than £20 000/QALY. For technologies with ICERs greater than £20 000/QALY, the number of decision-modifying factors was comparable for positive and negative recommendations.ConclusionsMost DAP decisions were consistent with the ICER threshold. However, cost-effectiveness was not the only determining factor in decision making; recommendations also considered patient- and healthcare-centric factors and uncertainty.     

Involving patients in decisions regarding preventive health interventions using the analytic hierarchy process

Practice guidelines that recommend active patient involvement in decisions about preventive health interventions are becoming increasingly common. These decisions frequently involve difficult trade‐offs between competing risks and benefits that require easily accessible information about the expected outcomes, superb doctor–patient communication, and effective integration of objective outcome data with individual values and preferences. Successful implementation of recommendations for shared decision‐making in preventive health care will require the development of efficient methods for making these complex decisions in busy practice settings. This article describes how the analytic hierarchy process, a multiple criteria decision‐making method, could facilitate successful implementation of shared decision‐making regarding preventive health care in clinical practice. The method is illustrated using recent guidelines for colorectal cancer screening for average risk patients issued by the American Gastroenterological Association.     

Health technology assessment for the NHS in England and Wales

In healthcare decision making, there is an important functional separation between assessment and appraisal. In the U.K. National Health Service (NHS), this distinction is illustrated by the separation of roles between the Health Technology Assessment Programme and the National Institute for Clinical Excellence. However, it can be seen at every level within the healthcare system. Assessment of a technology is a scientific task that synthesizes all relevant evidence on effectiveness and cost-effectiveness; its results are therefore generalizable. Appraisal of the technology is informed by the assessment but adds context-specific judgments on the applicability of the evidence, the feasibility and impact of alternative options, relative priorities, and wider social and ethical aspects. An explicit distinction between assessment and appraisal is helpful in achieving clarity, consistency, and consensus. It also makes clear the need for a wide range of assessment reviews to support decision making by commissioners, providers, and users of health services. Increasingly, the secondary research supported by the NHS R&D Programme is being distributed electronically. It is also being used to identify areas in which further primary research should be commissioned.     

"The future should not take us by surprise": preparation of an early warning system in Denmark

OBJECTIVES: To explore and test methods for the operation of a national Early Warning System (EWS) in Denmark and to support decision making by the Danish Centre for Evaluation and Health Technology Assessment on this issue. METHODS: On the basis of literature reviews, information from members of EuroScan, and supported by clinical experts and stakeholders, existing methods were adapted and new methods were developed as part of a feasibility study. RESULTS: Approximately 200 technologies in 30 specialties were identified on the basis of information by EuroScan. A new instrument was developed to distinguish between important and unimportant technologies (filtering). Clinical experts in six specialties applied the instrument to sixty-two technologies in their respective fields, of which nine (15%) were judged potentially important for the Danish health care system. For priority setting, adapting a Dutch instrument to the Danish context was discussed. In principle, the instrument was acceptable, but several changes were proposed, for example, relating to the decentralized structure of the Danish health care system. For early assessment, the format and methods applied by SBU and Canadian Coordinating Office for Health Technology Assessment (CCOHTA) were compared and applied to pharmaceuticals (glitazones in treatment of type 2 diabetes mellitus) and a procedure (embolization of uterine fibromas). Given the main target group of the Danish EWS, local decision makers, the CCOHTA format was preferred. CONCLUSIONS: The findings of the study have laid the foundation for an EWS using appropriate methods adapted to local circumstances. On the basis of the findings, a decision was made to start an EWS.     

Guía metodológica para la evaluación de la eficacia y la seguridad de nuevos fármacos: implementación de las recomendaciones de EUnetHTA

The European network for Health Technology Assessment (EUnetHTA) is the network of public health technology assessment (HTA) agencies and entities from across the EU. In this context, the HTA Core Model^®, has been developed. The Andalusian Agency for Health Technology Assessment (AETSA) is a member of the Spanish HTA Network and EUnetHTA collaboration In addition, AETSA participates in the new EUnetHTA Joint Action 3 (JA, 2016–2019). Furthermore, AETSA works on pharmaceutical assessments. Part of this work involves drafting therapeutic positioning reports (TPRs) on drugs that have recently been granted marketing authorisation, which is overseen by the Spanish Agency of Medicines and Medical Devices (AEMPS). AETSA contributes by drafting “Evidence synthesis reports: pharmaceuticals” in which a rapid comparative efficacy and safety assessment is performed for drugs for which a TPR will be created. To create this type of report, AETSA follows its own methodological guideline based on EUnetHTA guidelines and the HTA Core Model^®. In this paper, the methodology that AETSA has developed to create the guideline for “Evidence synthesis reports: pharmaceuticals” is described. The structure of the report itself is also presented.     

10 Years of End-of-Life Criteria in the United Kingdom

ObjectivesIn January 2009, the National Institute for Health and Care Excellence introduced supplementary guidance for end-of-life (EoL) treatments, which allowed treatments with an incremental cost-effectiveness ratio over the regular threshold (£20 000-£30 000) to be recommended, if they satisfied the EoL criteria. The aims of this study were (1) to systematically review 10 years of EoL supplementary guidance implementation and explore how it could be improved, and (2) to create a framework for incorporating the uncertainty relating to EoL criteria satisfaction into model-based cost-effectiveness analyses for decision making.MethodsAll appraisals between January 2009 and 2019 were screened for EoL discussions. Data were extracted on the EoL criteria and cost-effectiveness assessment details. Additionally, a quantitative method was developed to include the EoL criteria satisfaction uncertainty into model-based cost-effectiveness analyses. A stylized example was created to provide a case study for the inclusion of EoL criteria satisfaction uncertainty.ResultsAn EoL discussion was identified in 35% of appraisals, 57% of which led to a positive EoL decision. Only 5.7% of technologies with positive EoL decisions were not recommended, versus 43.8% of technologies with negative EoL decisions. EoL criteria assessment was often reported insufficiently and evaluated inconsistently and nontransparently. A total of 54.9% of EoL decisions were made while at least 1 criterion was surrounded by considerable uncertainty. By applying the proposed quantitative method, this EoL criteria satisfaction uncertainty was accounted for in decision making. The stylized example demonstrated that the impact of EoL criteria satisfaction uncertainty can be substantial enough to reverse the reimbursement decision.ConclusionsTo improve consistency/transparency and correct reimbursement decisions’ likelihood, new guidelines on the implementation of the EoL criteria are needed.     

The HTA Core Model®—10 Years of Developing an International Framework to Share Multidimensional Value Assessment

Background and Objectives

The HTA Core Model^® as a science-based framework for assessing dimensions of value was developed as a part of the European network for Health Technology Assessment project in the period 2006 to 2008 to facilitate production and sharing of health technology assessment (HTA) information, such as evidence on efficacy and effectiveness and patient aspects, to inform decisions.

Analytic hierarchy process for hospital site selection

Aim: This study investigated a decision support model for site selection to establish a new hospital based on the analytic hierarchy process (AHP). The main purpose of this study was to select the best site for a hospital using this process in Mu?la, Turkey.Method: AHP was employed as the methodological tool for the selection of the site. The study was based on 6 criteria and 19 sub-criteria. All districts in the province of Mu?la were evaluated as alternatives. These districts include: Bodrum, Dalaman, Datça, Fethiye, Kavakl?dere, Koyce?iz, Marmaris, Mente?e, Milas, Seydikemer, Ula, and Yata?an. The alternatives were ranked using a 1–9 Saaty scale. The analysis of the hierarchy model was conducted by using the Super Decisions 2.2.6 software program.Results: Results show that demand is the most important factor in determining the appropriate hospital site, followed by accessibility, competitors, government, related industry and environmental conditions. According to the results, Bodrum was chosen to be the best site to establish a new hospital.Conclusion: Due to limited resources in developing countries such as Turkey, it is critical that decisions are made as a result of scientific research. In this context, investors need to take into account some factors in line with this aim. The proposed evaluation criteria provide a reference for hospital administrators and investors in the selection of hospital sites using AHP.     

Creating a priority list of non-communicable diseases to support health research funding decision-making

ObjectiveTo develop and pilot a framework based on multi-criteria decision analysis (MCDA) to prioritize non-communicable diseases (NCDs) to support health research funding decision-making.MethodsThe framework involves identifying NCDs to be prioritized, specifying prioritization criteria and determining their weights from a survey of stakeholders. The mean weights from the survey are applied to the NCDs’ ratings on the criteria to generate a ‘total score’ for each NCD, by which the NCDs are prioritized.ResultsNineteen NCDs and five criteria were included. The criteria, in decreasing order of importance (mean weights in parentheses), are: deaths across the population (27.7 %), loss of quality-of-life across the population (23.0 %), cost to patients and families (18.6 %), cost to the health system (17.2 %), and whether vulnerable groups are disproportionately affected (13.4 %). The priority list of NCDs, stratified into four tiers of importance, is: ‘Very critical’ priority: coronary heart disease, back and neck pain, diabetes mellitus; ‘Critical’ priority: dementia and Alzheimer’s disease, stroke; ‘High’ priority: colon and rectum cancer, depressive disorders, chronic obstructive pulmonary disease, chronic kidney disease, breast cancer, prostate cancer, arthritis, lung cancer; and ‘Medium’ priority: asthma, hearing loss, melanoma skin cancer, addictive disorders, non-melanoma skin cancer, headaches.ConclusionThe results indicate the framework for prioritizing NCDs for research funding is feasible and effective. The framework could also be used for other health conditions.     

Systematic reviews published in higher impact clinical journals were of higher quality

ObjectivesTo compare the methodological quality of systematic reviews (SRs) published in high- and low–impact factor (IF) Core Clinical Journals. In addition, we aimed to record the implementation of aspects of reporting, including Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flow diagram, reasons for study exclusion, and use of recommendations for interventions such as Grading of Recommendations Assessment, Development and Evaluation (GRADE).Study Design and SettingWe searched PubMed for systematic reviews published in Core Clinical Journals between July 1 and December 31, 2012. We evaluated the methodological quality using the Assessment of Multiple Systematic Reviews (AMSTAR) tool.ResultsOver the 6-month period, 327 interventional systematic reviews were identified with a mean AMSTAR score of 63.3% (standard deviation, 17.1%), when converted to a percentage scale. We identified deficiencies in relation to a number of quality criteria including delineation of excluded studies and assessment of publication bias. We found that SRs published in higher impact journals were undertaken more rigorously with higher percentage AMSTAR scores (per IF unit: β = 0.68%; 95% confidence interval: 0.32, 1.04; P < 0.001), a discrepancy likely to be particularly relevant when differences in IF are large.ConclusionMethodological quality of SRs appears to be better in higher impact journals. The overall quality of SRs published in many Core Clinical Journals remains suboptimal.     

Joint project of the International Network of Agencies for Health Technology Assessment--Part 2: Managing the diffusion of positron emission tomography with health technology assessment

OBJECTIVES: Since 1997, members of the International Network of Agencies for Health Technology Assessment (INAHTA) have collaborated on a Joint Project to track the diffusion, evaluation, and clinical policy of positron emission tomography (PET). Part 2 of this updated Joint Project report summarizes HTA-based strategies for directing the clinical use of PET and a discussion on the value of HTA in managing the diffusion of high cost diagnostic technologies, which were presented at an INAHTA-sponsored workshop at the Health Technology Assessment International Annual Meeting in 2004 on strategies for managing high cost diagnostic technologies. METHODS: A summary of the workshop proceedings is presented. CONCLUSIONS: Sharing assessment work, universal agreement in assessment conclusions, stakeholder input, and modeling techniques help manage the uncertainty in the evidence base while targeting clinical use of PET toward the most promising indications. Emphasis on HTA findings, linkage between financing of clinical PET and outcome evaluation, and targeted dissemination of scientific findings empower providers to reduce unnecessary utilization and contain costs within a quality improvement framework. Above all, a trustworthy source of HTA information and a process that is conducive to using scientific evidence as the basis for decision making are essential for managing the diffusion of complex and costly diagnostic technologies in patient care.     

A Systematic Review of Research Guidelines in Decision-Analytic Modeling

BackgroundDecision-analytic modeling (DAM) has been increasingly used to aid decision making in health care. The growing use of modeling in economic evaluations has led to increased scrutiny of the methods used.ObjectiveThe objective of this study was to perform a systematic review to identify and critically assess good practice guidelines, with particular emphasis on contemporary developments.MethodsA systematic review of English language articles was undertaken to identify articles presenting guidance for good practice in DAM in the evaluation of health care. The inclusion criteria were articles providing guidance or criteria against which to assess good practice in DAM and studies providing criteria or elements for good practice in some areas of DAM. The review covered the period January 1990 to March 2014 and included the following electronic bibliographic databases: Cochrane Library, Cochrane Methodology Register and Health Technology Assessment, NHS Economic Evaluation Database, MEDLINE, and PubMed (Embase). Additional studies were identified by searching references.ResultsThirty-three articles were included in this review. A practical five-dimension framework was developed that describe the key elements of good research practice that should be considered and reported to increase the credibility of results obtained from DAM in the evaluation of health care.ConclusionsThis study is the first to critically review all available guidelines and statements of good practice in DAM since 2006. The development of good practice guidelines is an ongoing process, and important efforts have been made to identify what is good practice and to keep these guidelines up to date.     

The preferences and rationale of family doctors in pharmacological treatment for depression

Abstract

Background: Several systematic reviews of acupuncture as a treatment of insomnia have recently emerged. Their results are far from uniform. Aim: To summarize and critically evaluate these reviews with a view of defining the reasons for their discrepant conclusions and providing an overall verdict about the therapeutic value of acupuncture for insomnia. Methods: Thirteen electronic databases (Medline, Embase, Amed, CINHAL, Health Technology Assessments, DARE, Cochrane, six Korean/Chinese databases) were searched for relevant articles and data from the included reviews were extracted according to pre-defined criteria. Their methodological quality was assessed using the ‘Overview Quality Assessment Questionnaire’. Results: Ten systematic reviews of acupuncture for insomnia were published between 2003 and 2010. They differed in numerous respects. Several reviews draw strongly positive conclusions. Owing to these several caveats, the best evidence is, however, not clearly positive.

Conclusion: The evidence for acupuncture as a treatment of insomnia is plagued by important limitations, e.g. the poor quality of most primary studies and some systematic reviews. Those that are sensitive to such limitations, fail to arrive at a positive verdict about the effectiveness of acupuncture.     

Decision Aid Interventions for Family Caregivers of Persons With Advanced Dementia in Decision-Making About Feeding Options: A Scoping Review

ObjectivesWe provided an overview of the literature on decision aid interventions for family caregivers of older adults with advanced dementia regarding decision making about tube feeding. We synthesized (1) the use of theory during the development, implementation, and evaluation of decision aids; (2) the development, content, and delivery of decision aid interventions; (3) caregivers’ experience with decision aid interventions; and (4) the effect of decision aid interventions on caregivers’ quality of decision-making about feeding options.DesignScoping review.MethodsWe conducted a scoping review of peer-reviewed studies published January 1, 2000–June 30, 2022, in MEDLINE, EMBASE, The Cochrane Library, CINAHL, and Web of Science databases. The process was guided by Arksey and O’Malley’s methodological framework, which includes identifying the research question, choosing related studies, charting the data, and summarizing results. Empirical articles concerning the decision aid interventions about feeding options were selected.ResultsSix publications reporting 4 unique decision aid interventions were included. All the interventions targeted caregivers of older adults with advanced dementia. Three decision aids were culturally adapted from existing decision aids. The Ottawa Decision Support Framework and the International Patient Decision Aid Standards Framework were used in these 6 publications. Interventions aimed to improve decision making regarding tube feeding for caregivers through static delivery methods. Caregivers rated these decision aids as helpful and acceptable. Decisional conflict and knowledge of feeding options were the most common outcomes evaluated. Reduction in decisional conflict and increase in knowledge were consistently found among dementia caregivers, but no intervention effects were found on preferences for the use of tube feeding.Conclusions and ImplicationsDecision aid interventions effectively improve decision-making regarding tube feeding among the target population. Cultural adaptation of an existing decision aid intervention is the main strategy. However, the lack of guidance of a cultural adaptation framework in this process may lead to difficulties explaining caregivers’ behavioral changes. Moreover, merely providing information is not enough to change caregivers’ preferences or behavior of use of tube feeding. A systematic approach to cultural adaptation and interactive intervention is needed in future studies.     

Impact of the 1997 Canadian Guidelines on the Conduct of Canadian-Based Economic Evaluations in the Published Literature

ObjectiveTo assess the impact of the 1997 Canadian guidelines on the methods and presentation of economic evaluations conducted from a Canadian perspective in the published literature.MethodsA systematic literature review was conducted to identify health technology economic evaluations conducted from a Canadian perspective published in peer-reviewed journals between 2001 and 2006. To investigate the impact of the 1997 Canadian Coordinating Office of Health Technology Assessment guidelines, each included study was assessed against 17 of the 25 recommendations.ResultsOf the 153 included studies, a base set of 9 methodological standards, as outlined by the 1997 guidelines, were followed by over 50% of the studies including: indications, outcomes for cost utility analysis, outcomes for cost benefit analysis, discounting future cost and outcomes, cost identification and valuation, evaluating uncertainty and disclosing funding relationships. Main divergences from the guidelines were found for analytic technique (38%), study perspective (23%), source of preferences (8%), equity (7%), and cost measurement (24%).ConclusionThe current assessment has shown that the 1997 Canadian guidelines have set a minimum methodological standard within the community of “doers” conducting economic analyses from a Canadian perspective. Although there was divergence from some of the recommendations, the majority were reflected as changes in the 2006 Canadian guidelines.     

The Use of Multicriteria Decision Analysis to Support Decision Making in Healthcare: An Updated Systematic Literature Review

ObjectivesMulticriteria decision analysis (MCDA) is increasingly used for decision making in healthcare. However, its application in different decision-making contexts is still unclear. This study aimed to provide a comprehensive review of MCDA studies performed to inform decisions in healthcare and to summarize its application in different decision contexts.MethodsWe updated a systematic review conducted in 2013 by searching Embase, MEDLINE, and Google Scholar for MCDA studies in healthcare, published in English between August 2013 and November 2020. We also expanded the search by reviewing grey literature found via Trip Medical Database and Google, published between January 1990 and November 2020. A comprehensive template was developed to extract information about the decision context, criteria, methods, stakeholders involved, and sensitivity analyses conducted.ResultsFrom the 4295 identified studies, 473 studies were eligible for full-text review after assessing titles and abstracts. Of those, 228 studies met the inclusion criteria and underwent data extraction. The use of MCDA continues to grow in healthcare literature, with most of the studies (49%) informing priority-setting decisions. Safety, cost, and quality of care delivery are the most frequently used criteria, although there are considerable differences across decision contexts. Almost half of the MCDA studies used the linear additive model whereas scales and the analytical hierarchy process were the most used techniques for scoring and weighting, respectively. Not all studies report on each one of the MCDA steps, consider axiomatic properties, or justify the methods used.ConclusionsA guide on how to conduct and report MCDA that acknowledges the particularities of the different decision contexts and methods needs to be developed.