Assessing Differences in Utility Scores: A Comparison of Four Widely Used Preference-Based Instruments

OBJECTIVES: To characterize the differences in utility scores (dUTY) among four commonly used preference-based Health-Related Quality of Life instruments, to evaluate the potential impact of these differences on cost-utility analyses (CUA), and to determine if sociodemographic/clinical factors influenced the magnitude of these differences. METHODS: Consenting adult Chinese, Malay and Indian subjects in Singapore were interviewed using Singapore English, Chinese, Malay or Tamil versions of the EQ-5D, Health Utilities Index Mark 2 (HUI2) and Mark 3 (HUI3), and SF-6D. Agreement between instruments was assessed using Bland-Altman (BA) plots. Changes in incremental cost-utility ratio (ICUR) from dUTY were investigated using eight hypothetical decision trees. The influence of sociodemographic/clinical factors on dUTY between instrument pairs was studied using multiple linear regression (MLR) models for English-speaking subjects (circumventing structural zero issues). RESULTS: In 667 subjects (median age 48 years, 59% female), median utility scores ranged from 0.80 (95% confidence interval [CI] 0.80, 0.85) for the EQ-5D to 0.89 (95% CI 0.88, 0.89) for the SF-6D. BA plots: Mean differences (95% CI) exceeded the clinically important difference (CID) of 0.04 for four of six pairwise comparisons, with the exception of the HUI2/EQ-5D (0.03, CI: 0.02, 0.04) and SF-6D/HUI2 (0.02, CI: 0.006, 0.02). Decision trees: The ICER ranged from $94,661/QALY (quality-adjusted life-year; 6.3% difference from base case) to 100,693 dollars/QALY (0.3% difference from base case). MLR: Chronic medical conditions, marital status, and Family Functioning Measures scores significantly (P-value < 0.05) influenced dUTY for several instrument pairs. CONCLUSION: Although CIDs in utility measurements were present for different preference-based instruments, the impact of these differences on CUA appeared relatively minor. Chronic medical conditions, marital status, and family functioning influenced the magnitude of these differences.     

A comparison of generic, indirect utility measures (the HUI2, HUI3, SF-6D, and the EQ-5D) and disease-specific instruments (the RAQoL and the HAQ) in rheumatoid arthritis

Rheumatoid arthritis (RA) is a common, chronic disease where health-related quality of life (HRQL) is one of the main goals of therapy. As such, instruments used to measure HRQL in RA must be able to discriminate across RA severity. The two basic categories of instruments used to measure HRQL are generic instruments and disease-specific instruments. Generic instruments can be further subdivided into preference-based measures which yield both single and multi-attribute utility values anchored at zero (death) and 1.00 (perfect health) as a measure of HRQL. The scores from these types of instruments can be integrated into cost-utility analyses as the weightings for quality adjusted life years. We assessed the construct validity of utility scores from four generic preference-based measures (the Health Utilities Index 2 and 3 (HUI2, HUI3), the EuroQol 5D (EQ-5D), and the Short Form 6-D (SF-6D) and disease specific measures (the Rheumatoid Arthritis Quality of Life Questionnaire (RAQoL) and the Health Assessment Questionnaire (HAQ)) in a sample of 313 RA patients in British Columbia, Canada. We also estimated the minimally important differences (MID) for each of the measures. Generally, as anticipated, the disease-specific measures were better able to discriminate across groups with higher RA severity; however, utility scores from each of the scales also appeared to discriminate well across RA severity categories. The MID values agreed with those previously reported in the literature for the HUI2, SF-6D and the HAQ and provided new information for the HUI3, EQ-5D and the RAQoL. We conclude that the all of the preference-based utility measures that were evaluated appear to adequately discriminate across levels of RA severity.     

Responsiveness of generic health-related quality of life measures in stroke

Objective: To compare five preference-based generic measures of health-related quality of life (HRQOL) in terms of change scores, correlations among change scores, responsiveness, and quality adjusted life-years (QALYs) gained. Design: Observational longitudinal cohort study where clinical measures and self-assessed HRQOL measures were administered to stroke patients at baseline and at 6 months. Patients were categorized as ‘stable’, ‘some improvement’ and ‘large improvement’ using the Barthel Index, Modified Rankin Scale (MRS), and Center for Epidemiologic Studies Depression Scale (CES-D). For each group, paired t -tests and variants of effect size were used to compare the responsiveness of preference-based HRQOL summary scores, including the EQ-5D VAS and index-based score, SF-6D, and Health Utilities Index (HUI) Mark 2 (HUI2) and Mark 3 (HUI3) overall utility scores. Results: Ninety-eight of 124 (79%) patients completed the 6-month follow-up. Change scores of the EQ-Index, HUI2, and HUI3 were strongly correlated with changes in the Barthel Index and MRS, while the EQ-5D VAS had higher correlation with CES-D change scores than the other measures. The SF-6D, HUI3, and EQ-Index were generally more responsive than the HUI2 and EQ-5D Visual analogue scale (EQ-VAS). QALY estimates based on the EQ-5D index and HUI3 were twice as large as estimates based on the SF-6D and HUI2. Conclusions : The results of this study may assist in informing the selection of a preference-based generic HRQOL measure, although choice will also depend on study goals and context. We would caution against the generalization of the study results on responsiveness to conditions when more subtle change is expected.     

Comparing utility scores before and after hearing-aid provision

Utility scores were estimated for 609 hearing-impaired adults who completed EQ-5D, Health Utilities Index Mark III (HUI3) and SF-6D survey instruments both before and after being provided with a hearing aid. Pre-intervention, the mean utility scores for EQ-5D (0.80) and SF-6D (0.78) were significantly higher than the mean HUI3 utility score (0.58). Post-intervention, the mean improvement in the HUI3 (0.06 change) was significantly higher than the mean improvement according to the EQ-5D (0.01 change) or SF-6D (0.01 change). The estimated cost effectiveness of hearing-aid provision is therefore likely to be dependent on which instrument is used to measure utility.     

Are indirect utility measures reliable and responsive in rheumatoid arthritis patients?

Background: Preference-based, generic measures are increasingly being used to measure quality of life and as sources for quality weights in the estimation of Quality Adjusted Life Years (QALYs) in rheumatoid arthritis (RA). However, among the most commonly used instruments (the Health Utilities Index 2 and 3 [HUI2 and HUI3], the EuroQoL-5D [EQ-5D], and the Short Form-6D [SF-6D], there has been little comparative research. Therefore, we examined the reliability and responsiveness of these measures and the Rheumatoid Arthritis Quality of Life (RAQoL) and the Health Assessment Questionnaire (HAQ) in a sample of RA patients. Major findings: Test–retest reliability was acceptable for all of the instruments with the exception of the EQ-5D. Using two external criteria to define change (a patient transition question and categories of the patient global assessment of disease activity VAS), the RAQoL was the most responsive of the instruments. For the indirect utility instruments, the HUI3 and the SF-6D were the most responsive for measuring positive change. On average, for patients whose RA improved, the absolute change was highest for the HUI3. Conclusions: The HUI3 and the SF-6D appear to be the most responsive of the preference-based instruments in RA. However, differences in the magnitude of the absolute change scores have important implications for cost-effectiveness analyses.     

A view from the bridge: agreement between the SF-6D utility algorithm and the Health Utilities Index

BACKGROUND: The SF-6D is a new health state classification and utility scoring system based on 6 dimensions ('6D') of the Short Form 36, and permits a "bridging" transformation between SF-36 responses and utilities. The Health Utilities Index, mark 3 (HUI3) is a valid and reliable multi-attribute health utility scale that is widely used. We assessed within-subject agreement between SF-6D utilities and those from HUI3. METHODS: Patients at increased risk of sudden cardiac death and participating in a randomized trial of implantable defibrillator therapy completed both instruments at baseline. Score distributions were inspected by scatterplot and histogram and mean score differences compared by paired t-test. Pearson correlation was computed between instrument scores and also between dimension scores within instruments. Between-instrument agreement was by intra-class correlation coefficient (ICC). RESULTS: SF-6D and HUI3 forms were available from 246 patients. Mean scores for HUI3 and SF-6D were 0.61 (95% CI 0.60-0.63) and 0.58 (95% CI 0.54-0.62) respectively; a difference of 0.03 (p<0.03). Score intervals for HUI3 and SF-6D were (-0.21 to 1.0) and (0.30-0.95). Correlation between the instrument scores was 0.58 (95% CI 0.48-0.68) and agreement by ICC was 0.42 (95% CI 0.31-0.52). Correlations between dimensions of SF-6D were higher than for HUI3. CONCLUSIONS: Our study casts doubt on the whether utilities and QALYs estimated via SF-6D are comparable with those from HUI3. Utility differences may be due to differences in underlying concepts of health being measured, or different measurement approaches, or both. No gold standard exists for utility measurement and the SF-6D is a valuable addition that permits SF-36 data to be transformed into utilities to estimate QALYs. The challenge is developing a better understanding as to why these classification-based utility instruments differ so markedly in their distributions and point estimates of derived utilities.     

The Impact of Differences in EQ-5D and SF-6D Utility Scores on the Acceptability of Cost–Utility Ratios: Results across Five Trial-Based Cost–Utility Studies

ObjectiveThis article investigates whether differences in utility scores based on the EQ-5D and the SF-6D have impact on the incremental cost–utility ratios in five distinct patient groups.MethodsWe used five empirical data sets of trial-based cost–utility studies that included patients with different disease conditions and severity (musculoskeletal disease, cardiovascular pulmonary disease, and psychological disorders) to calculate differences in quality-adjusted life-years (QALYs) based on EQ-5D and SF-6D utility scores. We compared incremental QALYs, incremental cost–utility ratios, and the probability that the incremental cost–utility ratio was acceptable within and across the data sets.ResultsWe observed small differences in incremental QALYs, but large differences in the incremental cost–utility ratios and in the probability that these ratios were acceptable at a given threshold, in the majority of the presented cost–utility analyses. More specifically, in the patient groups with relatively mild health conditions the probability of acceptance of the incremental cost–utility ratio was considerably larger when using the EQ-5D to estimate utility. While in the patient groups with worse health conditions the probability of acceptance of the incremental cost–utility ratio was considerably larger when using the SF-6D to estimate utility.ConclusionsMuch of the appeal in using QALYs as measure of effectiveness in economic evaluations is in the comparability across conditions and interventions. The incomparability of the results of cost–utility analyses using different instruments to estimate a single index value for health severely undermines this aspect and reduces the credibility of the use of incremental cost–utility ratios for decision-making.     

Mapping Utility Scores from a Disease-Specific Quality-of-Life Measure in Bariatric Surgery Patients

Objectives:  To develop algorithms for a conversion of disease-specific quality-of-life into health state values for morbidly obese patients before or after bariatric surgery.
Methods:  A total of 893 patients were enrolled in a prospective cross-sectional multicenter study. In addition to demographic and clinical data, health-related quality-of-life (HRQoL) data were collected using the disease-specific Moorehead-Ardelt II questionnaire (MA-II) and two generic questionnaires, the EuroQoL-5D (EQ-5D) and the Short Form-6D (SF-6D). Multiple regression models were constructed to predict EQ-5D- and SF-6D-based utility values from MA-II scores and additional demographic variables.
Results:  The mean body mass index was 39.4, and 591 patients (66%) had already undergone surgery. The average EQ-5D and SF-6D scores were 0.830 and 0.699. The MA-IIwas correlated to both utility measures (Spearman's r  = 0.677 and 0.741). Goodness-of-fit was highest ( R ² = 0.55 in the validation sample) for the following item-based transformation algorithm: utility (MA-II-based) = 0.4293 + (0.0336 × MA1) + (0.0071 × MA2) + (0.0053 × MA3) + (0.0107 × MA4) + (0.0001 × MA5). This EQ-5D-based mapping algorithm outperformed a similar SF-6D-based algorithm in terms of mean absolute percentage error ( P  = 0.045).
Conclusions:  Because the mapping algorithm estimated utilities with only minor errors, it appears to be a valid method for calculating health state values in cost-utility analyses. The algorithm will help to define the role of bariatric surgery in morbid obesity.     

An assessment of validity and responsiveness of generic measures of health-related quality of life in hearing impairment

Purpose

This review examines psychometric performance of three widely used generic preference-based measures, that is, EuroQol 5 dimensions (EQ-5D), Health Utility Index 3 (HUI3) and Short-form 6 dimensions (SF-6D) in patients with hearing impairments.

Methods

A systematic search was undertaken to identify studies of patients with hearing impairments where health state utility values were measured and reported. Data were extracted and analysed to assess the reliability, validity (known group differences and convergent validity) and responsiveness of the measures across hearing impairments.

Results

Fourteen studies (18 papers) were included in the review. HUI3 was the most commonly used utility measures in hearing impairment. In all six studies, the HUI3 detected difference between groups defined by the severity of impairment, and four out of five studies detected statistically significant changes as a result of intervention. The only study available suggested that EQ-5D only had weak ability to discriminate difference between severity groups, and in four out of five studies, EQ-5D failed to detected changes. Only one study involved the SF-6D; thus, the information is too limited to conclude on its performance. Also evidence for the reliability of these measures was not found.

Conclusion

Overall, the validity and responsiveness of the HUI3 in hearing impairment was good. The responsiveness of EQ-5D was relatively poor and weak validity was suggested by limited evidence. The evidence on SF-6D was too limited to make any judgment. More head-to-head comparisons of these and other preference measures of health are required.     

Predicting EuroQoL EQ-5D preference scores from the SF-12 Health Survey in a nationally representative sample.

PURPOSE: To predict the EuroQoL EQ-5D utility index from the SF-12 Health Survey for a US national sample of adults. METHODS: The authors used the 2000 Medical Expenditure Panel Survey to examine the relationship between instruments. Linear regression was used to predict EQ-5D scores from Physical Component Summary (PCS) and Mental Component Summary (MCS) scores of the SF-12. A prediction model was derived in one half of the sample and validated in the other half. RESULTS: Complete responses to both measures were available for 14,580 adults; 7313 (50.2%) surveys were used for the derivation set. The 2-variable model predicted 61% of the variance in EQ-5D scores and provided reasonable ability to predict mean EQ-5D scores from mean PCS and MCS scores. Confidence intervals are dependent on sample size and variance of PCS and MCS scores. CONCLUSIONS: EQ-5D scores can be reasonably predicted from the SF-12. This model allows researchers to estimate utility data for use in decision and cost-utility analyses.     

Cost-effectiveness analysis of cochlear dose reduction by proton beam therapy for medulloblastoma in childhood

Background: The aim of this study is to evaluate the cost-effectiveness of proton beam therapy with cochlear dose reduction compared with conventional X-ray radiotherapy for medulloblastoma in childhood. Methods: We developed a Markov model to describe health states of 6-year-old children with medulloblastoma after treatment with proton or X-ray radiotherapy. The risks of hearing loss were calculated on cochlear dose for each treatment. Three types of health-related quality of life (HRQOL) of EQ-5D, HUI3 and SF-6D were used for estimation of quality-adjusted life years (QALYs). The incremental cost-effectiveness ratio (ICER) for proton beam therapy compared with X-ray radiotherapy was calculated for each HRQOL. Sensitivity analyses were performed to model uncertainty in these parameters. Results: The ICER for EQ-5D, HUI3 and SF-6D were $21 716/QALY, $11 773/QALY, and $20 150/QALY, respectively. One-way sensitivity analyses found that the results were sensitive to discount rate, the risk of hearing loss after proton therapy, and costs of proton irradiation. Cost-effectiveness acceptability curve analysis revealed a 99% probability of proton therapy being cost effective at a societal willingness-to-pay value. Conclusions: Proton beam therapy with cochlear dose reduction improves health outcomes at a cost that is within the acceptable cost-effectiveness range from the payer''s standpoint.     

Comparing Short Form 6D,Standard Gamble,and Health Utilities Index Mark 2 and Mark 3 utility scores: Results from total hip arthroplasty patients

Objective: The objectives are to compare SF-6D, standard gamble (SG), and Health Utilities Index (HUI) utility scores, compare change scores, and compare responsiveness. Methods: A cohort of osteoarthritis patients referred for total hip arthroplasty (THA) were evaluated at the time of referral and followed until 3months after THA. Patients were assessed using the SF-36, HUI2, HUI3, and the SG. Agreement is assessed using the intra-class correlation (ICC). Responsiveness is assessed using effect size, standardized response mean, and paired t-test. Results: Data was available for 86 patients at baseline and for 63 at both pre- and post-surgery. At baseline mean SF-6D (0.61), SG (0.62), and HUI2 (0.62) scores were similar; the mean HUI3 score (0.52) was lower. Standard deviations were 0.10, 0.32, 0.19, and 0.22. At baseline, agreement between SF-6D and SG scores was 0.13, agreement between SF-6D and HUI2 was 0.47, and agreement between SF-6D and HUI3 was 0.28. Agreement at pre- and post-surgery was similar. The change in scores between post- and pre-surgery was 0.10 for SF-6D, 0.16 for SG, 0.22 for HUI2, and 0.23 for HUI3. Effect sizes were 1.10 for HUI2, 1.08 for HUI3, 1.06 for SF-6D, and 0.48 for the SG. Conclusions: Agreement between SG scores and SF-6D and HUI scores was low. The estimate of change in utility associated with THA was lowest for SF-6D. Additional longitudinal studies to compare utility measures appear to be warranted.     

Mapping from Disease-Specific Measures to Utility: An Analysis of the Relationships between the Inflammatory Bowel Disease Questionnaire and Crohn''s Disease Activity Index in Crohn''s Disease and Measures of Utility

OBJECTIVES: To examine the relationship between the Inflammatory Bowel Disease Questionnaire (IBDQ), Crohn's Disease Activity Index (CDAI) and measures of utility (EQ-5D and the SF-6D indexes), and to estimate algorithms to map the two utility values from IBDQ and CDAI scores. METHODS: A large data set from clinical trials in Crohn's disease provided contemporaneous patient responses to all four questionnaires. Paired observations from multiple time-points were analyzed. We calculated mean utility scores by IBDQ and CDAI score deciles; Spearman correlation coefficients for paired observations between IBDQ and EQ-5D (n = 3320) and IBDQ and SF-6D (n = 3230), and explored regression models using maximum likelihood estimation. The IBDQ/SF-6D model was validated against paired observations from an independent data set. RESULTS: The IBDQ decile analysis demonstrated a consistent positive relationship with both utility indexes. Correlations between the IBDQ and both the EQ-5D and SF-6D were statistically significant (P < 0.0001), with correlation coefficients of 0.76 and 0.85, respectively. A simple linear model between EQ-5D and IBDQ explained 45% of the variance. The residuals plot for the IBDQ/SF-6D model suggested some nonlinearity and a nonlinear model explained 69% of the variance. In the validation analysis, no statistically significant difference was observed between the mean observed SF-6D and the SF-6D scores estimated using the IBDQ/SF-6D regression model. CONCLUSIONS: Given the strength, consistency, and predictable characteristics of the relationships, the algorithms appear to provide valuable and valid methods to estimate utilities from IBDQ scores (but not CDAI) in trials of Crohn's disease patients that have collected IBDQ scores but not utilities.     

EQ-5D Versus SF-12 in Coronary Patients: Are They Interchangeable?

ObjectivesThe aim of this study was to compare EuroQol five-dimensional (EQ-5D) utility scores and six-dimensional health state classification (SF-6D) utility scores (derived from the 12-Item Short-Form Health Survey [SF-12]) by using a large European sample of patients with stable coronary heart disease. Special attention was given to country-specific results.MethodsData from the EURopean Action on Secondary and Primary Prevention by Intervention to Reduce Events III (EUROASPIRE III) survey were used. Patients hospitalized for a coronary artery bypass graft, percutaneous coronary intervention, acute myocardial infarction, or myocardial ischemia were interviewed and examined at least 6 months after their acute event. Health-related quality of life was assessed by using the EQ-5D and the SF-12. SF-12 outcomes were converted to SF-6D utility values, allowing comparison between both measures.ResultsBoth EQ-5D and SF-6D results were available for 7472 patients with coronary heart disease from 20 European countries. The measures were significantly correlated (intraclass correlation coefficient = 0.536); however, large differences between the two measures remain. A total of 28.8% of the patients reported a ceiling effect on the EQ-5D instrument, whereas only 4.2% of the patients reported full health based on the SF-6D. Especially the mental component does not seem to be completely captured by the EQ-5D instrument. Furthermore, patients with worse EQ-5D outcomes were more likely to have better SF-6D results, whereas patients with better EQ-5D outcomes were more likely to have worse SF-6D results.ConclusionsBoth measures are not interchangeable. Whereas the main disadvantage of the EQ-5D questionnaire is its ceiling effect, the potential advantages of SF-12 might disappear when converting the outcomes into an SF-6D utility, because of the small differences between patients.     

Unchained melody: revisiting the estimation of SF-6D values

Purpose

In the original SF-6D valuation study, the analytical design inherited conventions that detrimentally affected its ability to predict values on a quality-adjusted life year (QALY) scale. Our objective is to estimate UK values for SF-6D states using the original data and multi-attribute utility (MAU) regression after addressing its limitations and to compare the revised SF-6D and EQ-5D value predictions.

Methods

Using the unaltered data (611 respondents, 3503 SG responses), the parameters of the original MAU model were re-estimated under three alternative error specifications, known as the instant, episodic, and angular random utility models. Value predictions on a QALY scale were compared to EQ-5D3L predictions using the 1996 Health Survey for England.

Results

Contrary to the original results, the revised SF-6D value predictions range below 0 QALYs (i.e., worse than death) and agree largely with EQ-5D predictions after adjusting for scale. Although a QALY is defined as a year in optimal health, the SF-6D sets a higher standard for optimal health than the EQ-5D-3L; therefore, it has larger units on a QALY scale by construction (20.9 % more).

Conclusions

Much of the debate in health valuation has focused on differences between preference elicitation tasks, sampling, and instruments. After correcting errant econometric practices and adjusting for differences in QALY scale between the EQ-5D and SF-6D values, the revised predictions demonstrate convergent validity, making them more suitable for UK economic evaluations compared to original estimates.

     

Interchangeability of the EQ-5D and the SF-6D in Long-Lasting Low Back Pain

Objectives:  The objective of this study was to investigate the interchangeability of the EuroQol 5D (EQ-5D) and the Short Form 6D (SF-6D) in individuals with long-lasting low back pain to guide the optimal choice of instrument and to inform decision-makers about any between-measure discrepancy, which require careful interpretation of the results of cost-utility evaluations.
Methods:  A cross-sectional study was conducted across 275 individuals who had spinal surgery on indication of chronic low back pain. EQ-5D and SF-6D were mailed to respondents for self-completion. Statistical analysis of between-measure agreement (using English weights) was based on Bland and Altman's limits of agreement and a series of linear regressions.
Results:  A moderate mean difference of 0.085 (SD 0.241) was found, but because it masked more severe bidirectional variation, the expected variation between observations of EQ-5D and SF-6D in future studies was estimated at 0.546. The EQ-5D's N3 term alone explained a factor of 0.79 of the variation in between-measure differences, while the explanatory value of adding variables of age, sex, diagnosis, previous surgery, and occupational status was basically zero. A final model including only dummy variables for the N3 term and five identified framing effects explained a factor of 0.86 of the variation in between-measure differences.
Conclusions:  Although the EQ-5D and the SF-6D are both psychometrically valid for generic outcome assessment in long-lasting low back pain, it appears that they cannot generally be used interchangeably for measurement of preference values. Sensitivity analysis examining the impact of between-measure discrepancy thus remains a necessary condition for the interpretation of the results of cost-utility evaluations.     

Evaluating quality-adjusted life years: estimation of the health utility index (HUI2) from the SF-36.

Quality-adjusted life years (QALYs) are well recognized as a valid measure for outcomes in cost-effectiveness analyses. A summary health utility score is necessary to evaluate QALYs. The objective of this study was to predict a summary utility score (represented by the Health Utility Index [HUI2]) from scores on the SF-36. METHODS: A structural equation framework was applied to longitudinal data collected from 1992 to 1995 on a sample of patients insured by Southem California Kaiser Permanente (N = 6921). An ordinary least squares (OLS) method was used to estimate the HUI2. RESULTS: The OLS model on cross-sectional data predicted 50.5% of the observed variance in HUI2 scores. Parameter estimates of all SF-36 components showed statistical significance at the P < 0.05 level. CONCLUSIONS: Results of this study provide a quantitative link between two important measures of health status. The present model can be used to estimate health utility summary scores in studies that have collected SF-36 data.