Assessing generalisability by location in trial-based cost-effectiveness analysis: the use of multilevel models

Cost-effectiveness analysis (CEA) in health care is increasingly conducted alongside multicentre and multinational randomised controlled clinical trials (RCTs). The increased use of stochastic CEA is designed to account for between-patient sampling variability in cost-effectiveness data assuming that observations are independently distributed. However, between-location variability in cost-effectiveness may result if there is a hierarchical structure in the data; that is, if there is correlation in costs and outcomes between patients recruited in particular locations. This may be expected in multi-location trials given that centres and countries often differ in factors such as clinical practice, patient case-mix and the unit costs of delivering health care. A failure to acknowledge this feature may lead to misleading conclusions in a trial-based economic study. Multilevel modelling (MLM) is an analytical framework that can be used to handle hierarchical cost-effectiveness data. Using data from a recently conducted economic analysis, this paper shows how multilevel modelling can be used to obtain (a) more appropriate estimates of the population average incremental cost-effectiveness and associated standard errors compared to standard stochastic CEA; and (b) location-specific estimates of incremental cost-effectiveness which can be used to explore appropriately the variability between centres/countries of the cost-effectiveness results.     

Selecting a decision model for economic evaluation: a case study and review

The increased use of modelling techniques as a methodological tool in the economic evaluation of health care technologies has, in the main, been limited to two approaches – decision trees and Markov chain models. The former are suited to modelling simple scenarios that occur over a short time period, whilst Markov chain models allow longer time periods to be modelled, in continuous time, where the timing of an event is uncertain. In the context of economic evaluation, a less well developed technique is discrete event simulation, which may allow even greater flexibility. Taking the economic evaluation of adjuvant therapies for breast cancer as an illustrative example, the process of building a decision tree, a Markov chain model, and a discrete event simulation model are described. The potential benefits and problems of each approach are discussed. The suitability of the modelling techniques to economic evaluations of health care programmes in general is then discussed. This section aims to illustrate the areas in which the alternative modelling methods may be most appropriately employed. This revised version was published online in August 2006 with corrections to the Cover Date.     

Job load and hazard analysis: a method for the analysis of workplace conditions for occupational health care

One requirement for successful occupational health care is reliable information on occupational hazards. The aim of this study was to develop a simple, standardised method for workplace investigations for use in occupational health care. The theoretical framework of the method comprises the stress-strain model, the hazard-danger model, and risk behaviour theory. The new method, termed job load and hazard analysis, includes four stages: identification of hazards, their evaluation, conclusions and proposals, and follow up. Different methods are available for hazard identification. The identification starts with a rough analysis of five factors, chemical hazards, physical hazards, physical load, mental stress, and accident risk. Hazards and stress factors are assessed with an ordinal scale. Specialised methods are used if all hazards cannot otherwise be identified. The analytical procedure comprises: detection of hazards through observations and interviews at the workplace and with a questionnaire; assessment of findings as teamwork; and evaluation of the results of these assessments to yield conclusions and proposals made by occupational health care personnel. A data processing system has been developed for data storage and future use. The method has functioned in practice, improving the contents of the occupational health care programme and generating preventive measures. The method offers many new possibilities for controlling occupational hazards and studying relations between working conditions and workers' health.     

Job load and hazard analysis: a method for the analysis of workplace conditions for occupational health care.

One requirement for successful occupational health care is reliable information on occupational hazards. The aim of this study was to develop a simple, standardised method for workplace investigations for use in occupational health care. The theoretical framework of the method comprises the stress-strain model, the hazard-danger model, and risk behaviour theory. The new method, termed job load and hazard analysis, includes four stages: identification of hazards, their evaluation, conclusions and proposals, and follow up. Different methods are available for hazard identification. The identification starts with a rough analysis of five factors, chemical hazards, physical hazards, physical load, mental stress, and accident risk. Hazards and stress factors are assessed with an ordinal scale. Specialised methods are used if all hazards cannot otherwise be identified. The analytical procedure comprises: detection of hazards through observations and interviews at the workplace and with a questionnaire; assessment of findings as teamwork; and evaluation of the results of these assessments to yield conclusions and proposals made by occupational health care personnel. A data processing system has been developed for data storage and future use. The method has functioned in practice, improving the contents of the occupational health care programme and generating preventive measures. The method offers many new possibilities for controlling occupational hazards and studying relations between working conditions and workers' health.     

Systematic reviews of economic evaluations: utility or futility?

Systematic reviews of studies of effectiveness are the centrepiece of evidence‐based medicine and policy making. Increasingly, systematic reviews of economic evaluations are also an expected input into much evidence‐based policy making, with some health economists even calling for ‘an economics approach to systematic review’. This paper questions the value of conducting systematic reviews of economic evaluations to inform decision making in health care. It argues that the value of systematic reviews of economic evaluations is usually undermined by three things. Firstly, compared with effectiveness studies, there is a much wider range of factors that limit the generalisability of cost–effectiveness results, over time and between health systems and service settings, including the context‐dependency of resource use and opportunity costs, and different decision contexts and budget constraints. Secondly, because economic evaluations are more explicitly intended to be decision‐informing, the requirements for generalisability take primacy, and considerations of internal validity become more secondary. Thirdly, since one of the two main forms of economic evaluation – decision analytic modelling – is itself a well‐developed method of evidence synthesis, in most cases the need for a comprehensive systematic review of previous economic evaluations of a particular health technology or policy choice is unwarranted. I conclude that apparent ‘meta‐analytic expectations’ for clear and widely applicable cost–effectiveness conclusions from systematic reviews of economic evaluations are optimistic and generally futile. For more useful insights and knowledge from previous economic studies in evidence‐based policy making, a more limited range of reasons for conducting systematic reviews of health economic studies is proposed. Copyright © 2009 John Wiley & Sons, Ltd.     

A taxonomy of model structures for economic evaluation of health technologies

Models for the economic evaluation of health technologies provide valuable information to decision makers. The choice of model structure is rarely discussed in published studies and can affect the results produced. Many papers describe good modelling practice, but few describe how to choose from the many types of available models. This paper develops a new taxonomy of model structures. The horizontal axis of the taxonomy describes assumptions about the role of expected values, randomness, the heterogeneity of entities, and the degree of non-Markovian structure. Commonly used aggregate models, including decision trees and Markov models require large population numbers, homogeneous sub-groups and linear interactions. Individual models are more flexible, but may require replications with different random numbers to estimate expected values. The vertical axis of the taxonomy describes potential interactions between the individual actors, as well as how the interactions occur through time. Models using interactions, such as system dynamics, some Markov models, and discrete event simulation are fairly uncommon in the health economics but are necessary for modelling infectious diseases and systems with constrained resources. The paper provides guidance for choosing a model, based on key requirements, including output requirements, the population size, and system complexity.     

Modelling in the economic evaluation of health care: selecting the appropriate approach

OBJECTIVES: To provide an overview of alternative approaches to modelling in economic evaluation, and to highlight situations where each of the alternative modelling techniques should be employed. METHODS: A review of the available approaches to modelling in the economic evaluation of health care interventions with a leading discussion of examples of published studies leading to guidance in the selection of an appropriate approach in different circumstances. RESULTS: The main approaches to modelling used in economic evaluations in health care are decision trees, Markov models and individual sampling models. These methods assume independence of individuals within the model. Where interaction between individuals is important, other methods such as discrete-event simulation or system dynamics are preferable. CONCLUSIONS: The paper highlights the crucial question to be answered when selecting the approach to modelling: can the individuals being simulated in the model be regarded as independent? This issue is very commonly not recognised by analysts but is fundamental to the appropriate application of modelling in economic evaluation.     

Cognitive Functional Evaluation (CFE) Process for Individuals with Suspected Cognitive Disabilities

The purpose of this paper is to conceptualize the evaluation process for individuals with suspected cognitive disabilities. The Cognitive Functional Evaluation (CFE) process yields a comprehensive profile of the clients' cognitive strengths and weaknesses in occupational performance. The components of the CFE are outlined in six stages as a decision tree with examples of standardized instruments from which to choose the assessments for each client evaluated: (1) interview and background information; (2) cognitive screening and baseline status tests; (3) general measures of cognition and executive functions in occupation; (4) cognitive tests for specific domains; (5) measures of specific cognitive domains in occupations; and (6) environmental assessment. The first three stages are required to ascertain basic cognitive abilities underlying occupational performance. Tests for each stage can be chosen from the ones listed according to the client characteristics and the theory utilized, there is no need to use all of them. Once this data is available a further decision is made whether a more in-depth assessment is needed (stages (4) and (5)). The environmental component is evaluated in all instances with at least one of the assessments. The CFE process for individuals with suspected cognitive disabilities is recommended to be used by occupational therapists as a common ground for evaluation, documentation, and communicating information.     

An integrated comprehensive occupational surveillance system for health care workers

BACKGROUND: Workers in the health care industry may be exposed to a variety of work-related stressors including infectious, chemical, and physical agents; ergonomic hazards; psychological hazards; and workplace violence. Many of these hazards lack surveillance systems to evaluate exposures and health outcomes. The development and implementation of a comprehensive surveillance system within the Duke University Health System (DUHS) that tracks occupational exposures and stressors as well as injuries and illnesses among a defined population of health care workers (HCWs) is presented. METHODS: Human resources job and work location data were used to define the DUHS population at risk. Outcomes and exposure data from existing occupational health and safety programs, health promotion programs, and employee health insurance claims, were linked with human resources data and de-identified to create the Duke Health and Safety Surveillance System (DHSSS). RESULTS: The surveillance system is described and four examples are presented demonstrating how the system has successfully been used to study consequences of work-related stress, hearing conservation program evaluation, risk factors for back pain and inflammation, and exposures to blood and body fluids (BBF). CONCLUSIONS: Utilization of existing data, often collected for other purposes, can be successfully integrated and used for occupational health surveillance monitoring of HCWs. Use of the DHSSS for etiologic studies, benchmarking, and intervention program evaluation are discussed.     

Model Performance Evaluation (Validation and Calibration) in Model-based Studies of Therapeutic Interventions for Cardiovascular Diseases

Decision analytic models play an increasingly important role in the economic evaluation of health technologies. Given uncertainties around the assumptions used to develop such models, several guidelines have been published to identify and assess ‘best practice’ in the model development process, including general modelling approach (e.g., time horizon), model structure, input data and model performance evaluation. This paper focuses on model performance evaluation. In the absence of a sufficient level of detail around model performance evaluation, concerns regarding the accuracy of model outputs, and hence the credibility of such models, are frequently raised. Following presentation of its components, a review of the application and reporting of model performance evaluation is presented. Taking cardiovascular disease as an illustrative example, the review investigates the use of face validity, internal validity, external validity, and cross model validity. As a part of the performance evaluation process, model calibration is also discussed and its use in applied studies investigated. The review found that the application and reporting of model performance evaluation across 81 studies of treatment for cardiovascular disease was variable. Cross-model validation was reported in 55 % of the reviewed studies, though the level of detail provided varied considerably. We found that very few studies documented other types of validity, and only 6 % of the reviewed articles reported a calibration process. Considering the above findings, we propose a comprehensive model performance evaluation framework (checklist), informed by a review of best-practice guidelines. This framework provides a basis for more accurate and consistent documentation of model performance evaluation. This will improve the peer review process and the comparability of modelling studies. Recognising the fundamental role of decision analytic models in informing public funding decisions, the proposed framework should usefully inform guidelines for preparing submissions to reimbursement bodies.     

Meta‐analysis of Gaussian individual patient data: Two‐stage or not two‐stage?

Quantitative evidence synthesis through meta‐analysis is central to evidence‐based medicine. For well‐documented reasons, the meta‐analysis of individual patient data is held in higher regard than aggregate data. With access to individual patient data, the analysis is not restricted to a “two‐stage” approach (combining estimates and standard errors) but can estimate parameters of interest by fitting a single model to all of the data, a so‐called “one‐stage” analysis. There has been debate about the merits of one‐ and two‐stage analysis. Arguments for one‐stage analysis have typically noted that a wider range of models can be fitted and overall estimates may be more precise. The two‐stage side has emphasised that the models that can be fitted in two stages are sufficient to answer the relevant questions, with less scope for mistakes because there are fewer modelling choices to be made in the two‐stage approach. For Gaussian data, we consider the statistical arguments for flexibility and precision in small‐sample settings. Regarding flexibility, several of the models that can be fitted only in one stage may not be of serious interest to most meta‐analysis practitioners. Regarding precision, we consider fixed‐ and random‐effects meta‐analysis and see that, for a model making certain assumptions, the number of stages used to fit this model is irrelevant; the precision will be approximately equal. Meta‐analysts should choose modelling assumptions carefully. Sometimes relevant models can only be fitted in one stage. Otherwise, meta‐analysts are free to use whichever procedure is most convenient to fit the identified model.     

EFFICIENCY AND EQUITY: A STATED PREFERENCE APPROACH

Outcome measurement in the economic evaluation of health care considers outcomes independent of to whom they accrue. This article reports on a discrete choice experiment designed to elicit population preferences regarding the allocation of health gain between hypothetical groups of potential patients. A random‐effects probit model is estimated, and a technique for converting these results into equity weights for use in economic evaluation is adopted. On average, the modelling predicts a relatively high social value on health gains accruing to nonsmokers, carers, those with a low income and those with an expected age of death less than 45 years. Respondents tend to favour individuals with similar characteristics to themselves. These results challenge the conventional practice of assuming constant equity weighting. For decision makers, whether a formal equity weighting system represents an improvement on more informal approaches to weighing up equity and efficiency concerns remains uncertain. Copyright © 2012 John Wiley & Sons, Ltd.     

Psychosocial factors influencing non-urgent use of the emergency room: a review of the literature and recommendations for research and improved service delivery.

Despite dramatic increases in use of hospital emergency rooms (ERs) since the 1950s, an estimated 85% of ER visits are made for non-life-threatening reasons. Using a modified version of the Andersen and Newman model of health care utilization, this paper reviews the research literature on ER use to examine what is known about factors that influence three stages of the help-seeking process: (1) problem recognition; (2) the decision to seek help; (3) the decision to use the ER. While predisposing factors other than race are not generally significant, enabling factors such as income, insurance coverage, having a usual source of care, and geographic proximity affect use of the ER, both alone and in interaction with race and other factors. The most common reason for non-urgent ER use was 'other care not available'. In addition to the absence of primary care, non-urgent use of the ER is linked to need factors arising from socioeconomic stress, psychiatric co-morbidities, and a lack of social support. Recommendations for future studies include examining prospectively all 3 stages of decision-making leading to ER use, and considering interactive effects among predictors. Implications for service delivery are discussed, including the need to re-structure health care delivery systems to provide greater access to primary care and provide more attention to psychosocial aspects of patient care in clinical settings.     

A Health Economics Approach to US Value Assessment Frameworks—Summary and Recommendations of the ISPOR Special Task Force Report [7]

This summary section first lists key points from each of the six sections of the report, followed by six key recommendations. The Special Task Force chose to take a health economics approach to the question of whether a health plan should cover and reimburse a specific technology, beginning with the view that the conventional cost-per-quality-adjusted life-year metric has both strengths as a starting point and recognized limitations. This report calls for the development of a more comprehensive economic evaluation that could include novel elements of value (e.g., insurance value and equity) as part of either an “augmented” cost-effectiveness analysis or a multicriteria decision analysis. Given an aggregation of elements to a measure of value, consistent use of a cost-effectiveness threshold can help ensure the maximization of health gain and well-being for a given budget. These decisions can benefit from the use of deliberative processes. The six recommendations are to: 1) be explicit about decision context and perspective in value assessment frameworks; 2) base health plan coverage and reimbursement decisions on an evaluation of the incremental costs and benefits of health care technologies as is provided by cost-effectiveness analysis; 3) develop value thresholds to serve as one important input to help guide coverage and reimbursement decisions; 4) manage budget constraints and affordability on the basis of cost-effectiveness principles; 5) test and consider using structured deliberative processes for health plan coverage and reimbursement decisions; and 6) explore and test novel elements of benefit to improve value measures that reflect the perspectives of both plan members and patients.     

Why do the sick not utilise health care? The case of Zambia

When ill the individual faces the options of seeking health care, using self-medication or doing nothing. In an economic perspective, an individual's propensity to utilise health care is determined by the costs of utilisation and the perceived benefits of health care. The propensity to utilise health care may hence be expected to vary between individuals. In this paper we attempt on the one hand to determine what factors influence sick individuals' propensity to seek health care at a health facility or use self-medication (or do nothing), and on the other hand attempt to determine the factors that influence the magnitude of their expenditures for health care, in particular what other factors than just health status influence utilisation. For the empirical analysis, data, covering 9700 individuals, from the 1998 Living Conditions Monitoring Survey (LCMS) is used. We use a Multinomial Logit selection model to estimate the equation, which allows us to analyse health-care utilisation through two separate processes, the decision to seek care and the magnitude of expenditures incurred. In general, we find that the individuals are influenced by income, insurance, type of illness and access variables such as distance and owning a vehicle.     

Improving the quality of decision-making in health care management: a project report from the Nevada/Bohemia health management education partnership.

Health care is a data-sensitive and data-rich industry. Designers of health administration curricula recognize that future health care providers and managers must be trained to use available analytical and database information technology (IT) to transform these data into information helpful to the decision-maker. However, as the data available to administrators and clinicians proliferates, additional skills are needed to select data that is pertinent and useful. This paper presents the results of the collaboration of partners from Bohemia and University of Nevada and focuses on three areas: threats to the effective use of data to support health care decision-making;. health care decision support research and training strategies; future cross-disciplinary collaboration in health care decision support. Advanced IT methods have the potential to assist clinical and managerial decision-making. If further discussion is stimulated about the introduction of advanced IT methods into the health care management curriculum and research agenda, this paper will achieve its goal.     

Mapping to Estimate Health-State Utility from Non–Preference-Based Outcome Measures: An ISPOR Good Practices for Outcomes Research Task Force Report

Economic evaluation conducted in terms of cost per quality-adjusted life-year (QALY) provides information that decision makers find useful in many parts of the world. Ideally, clinical studies designed to assess the effectiveness of health technologies would include outcome measures that are directly linked to health utility to calculate QALYs. Often this does not happen, and even when it does, clinical studies may be insufficient for a cost-utility assessment. Mapping can solve this problem. It uses an additional data set to estimate the relationship between outcomes measured in clinical studies and health utility. This bridges the evidence gap between available evidence on the effect of a health technology in one metric and the requirement for decision makers to express it in a different one (QALYs). In 2014, ISPOR established a Good Practices for Outcome Research Task Force for mapping studies. This task force report provides recommendations to analysts undertaking mapping studies, those that use the results in cost-utility analysis, and those that need to critically review such studies. The recommendations cover all areas of mapping practice: the selection of data sets for the mapping estimation, model selection and performance assessment, reporting standards, and the use of results including the appropriate reflection of variability and uncertainty. This report is unique because it takes an international perspective, is comprehensive in its coverage of the aspects of mapping practice, and reflects the current state of the art.     

A framework for developing evaluation capacity in health care settings.

The purpose of this paper is to provide a framework for developing an effective evaluation practice within health care settings. Three features are reviewed; capacity building, the application of evaluation to program activities and the utilization of evaluation recommendations. First, the organizational elements required to establish effective evaluation practice are reviewed emphasizing that an organization's capacity for evaluation develops over time and in stages. Second, a comprehensive evaluation framework is presented which demonstrates how evaluation practice can be applied to all aspects of a program's life cycle, thus promoting the scope of evidence-based decision making within an organization. Finally, factors which influence the adoption of evaluation recommendations by decision makers are reviewed accompanied by strategies to promote the utilization of evaluation recommendations in organization decision making.     

卫生决策支持系统设计与实现

本文基于健康医疗数据资源,以提升卫生决策水平为目的,设计了覆盖公共卫生、医疗服务、基层卫生、药品管理、医疗保障等业务领域的具备信息支持、监控预警、分析评价、趋势预测、综合优化等核心功能的卫生决策支持系统,以及包含数据层、业务层、表现层的系统架构和数据仓库、模型库、方法库、知识库等决策支持库;并以医疗保障(新农合)业务的决策支持应用为例,对部分决策支持的功能进行开发应用;最后提出了构建卫生决策支持系统的建议。