A view from the bridge: agreement between the SF-6D utility algorithm and the Health Utilities Index

BACKGROUND: The SF-6D is a new health state classification and utility scoring system based on 6 dimensions ('6D') of the Short Form 36, and permits a "bridging" transformation between SF-36 responses and utilities. The Health Utilities Index, mark 3 (HUI3) is a valid and reliable multi-attribute health utility scale that is widely used. We assessed within-subject agreement between SF-6D utilities and those from HUI3. METHODS: Patients at increased risk of sudden cardiac death and participating in a randomized trial of implantable defibrillator therapy completed both instruments at baseline. Score distributions were inspected by scatterplot and histogram and mean score differences compared by paired t-test. Pearson correlation was computed between instrument scores and also between dimension scores within instruments. Between-instrument agreement was by intra-class correlation coefficient (ICC). RESULTS: SF-6D and HUI3 forms were available from 246 patients. Mean scores for HUI3 and SF-6D were 0.61 (95% CI 0.60-0.63) and 0.58 (95% CI 0.54-0.62) respectively; a difference of 0.03 (p<0.03). Score intervals for HUI3 and SF-6D were (-0.21 to 1.0) and (0.30-0.95). Correlation between the instrument scores was 0.58 (95% CI 0.48-0.68) and agreement by ICC was 0.42 (95% CI 0.31-0.52). Correlations between dimensions of SF-6D were higher than for HUI3. CONCLUSIONS: Our study casts doubt on the whether utilities and QALYs estimated via SF-6D are comparable with those from HUI3. Utility differences may be due to differences in underlying concepts of health being measured, or different measurement approaches, or both. No gold standard exists for utility measurement and the SF-6D is a valuable addition that permits SF-36 data to be transformed into utilities to estimate QALYs. The challenge is developing a better understanding as to why these classification-based utility instruments differ so markedly in their distributions and point estimates of derived utilities.     

Evaluating quality-adjusted life years: estimation of the health utility index (HUI2) from the SF-36.

Quality-adjusted life years (QALYs) are well recognized as a valid measure for outcomes in cost-effectiveness analyses. A summary health utility score is necessary to evaluate QALYs. The objective of this study was to predict a summary utility score (represented by the Health Utility Index [HUI2]) from scores on the SF-36. METHODS: A structural equation framework was applied to longitudinal data collected from 1992 to 1995 on a sample of patients insured by Southem California Kaiser Permanente (N = 6921). An ordinary least squares (OLS) method was used to estimate the HUI2. RESULTS: The OLS model on cross-sectional data predicted 50.5% of the observed variance in HUI2 scores. Parameter estimates of all SF-36 components showed statistical significance at the P < 0.05 level. CONCLUSIONS: Results of this study provide a quantitative link between two important measures of health status. The present model can be used to estimate health utility summary scores in studies that have collected SF-36 data.     

Validity, Responsiveness, and Minimal Important Difference for the SF-6D Health Utility Scale in a Spinal Cord Injured Population

Objective:  To determine the feasibility, acceptability, discriminative validity, responsiveness, and minimal important difference (MID) of the SF-6D for people with spinal cord injury (SCI).
Methods:  A total of 305 people with SCI completed the SF-36 health status questionnaire at baseline and at subsequent occurrence of a urinary tract infection (UTI) or 6-month follow-up. Normative SF-36 data were obtained from the Australian Bureau of Statistics. SF-36 scores were transformed to SF-6D utility values using Brazier's algorithm. We used UTI as the external criterion of clinically important change to determine responsiveness and two categories of the SF-36 transition question ("somewhat worse" and "somewhat better") as the external criterion to determine the MID. Derived SF-12 responsiveness was also assessed.
Results:  The mean SF-6D values were: 0.68 (SD 0.21, n = 305) all patients; 0.66 (SD 0.19, n = 167) tetraplegia; 0.72 (SD 0.26, n = 138) paraplegia; 0.57 (SD 0.15, n = 138) with UTI. The Australian normative SF-6D mean value was 0.80 (SD 0.14, n = 18,005). The SF-6D was able to discriminate between SCI and the Australian normative sample (effect size [ES] = 0.86), tetraplegia–paraplegia (ES = 0.23), and it was responsive to UTI (ES = 0.86 SF-36 variant, ES = 0.92 SF-12 variant). The MID for respondents who reported being somewhat worse or somewhat better at follow-up was 0.03 (SD 0.17, n = 108/305), while the MID for only those who were somewhat worse was 0.10 (SD 0.14, n = 58).
Conclusions:  The content of the SF-6D is more appropriate than that of the SF-36 for this physically impaired population. The SF-6D has discriminative power and is responsive to clinically important change because of UTI. The MID is consistent with published estimates for other disease groups.     

Comparing the Health Utilities Index Mark 3 (HUI3) with the Short Form-36 Preference-Based SF-6D in Chronic Kidney Disease

Objective:  Assess within-subject agreement and compare discriminative abilities between the SF-6D and the Health Utilities Index Mark 3 (HUI3) in patients with chronic kidney disease (CKD).
Methods:  The HUI3 and Short Form-36 were self-completed by 185 CKD patients enrolled in a prospective study of incident patients with stage 4 and 5 CKD.
Results:  The mean preference-based score for the SF-6D was 0.67 ± 0.13 compared to 0.58 ± 0.26 for the HUI3 ( P  < 0.01). There was a strong association between SF-6D and HUI3 scores (Pearson correlation coefficient 0.55, 95% CI 0.43–0.65) and moderate agreement with an intraclass correlation coefficient of 0.44. The HUI3 was better able to capture more severe burden of illness with fewer floor effects. The SF-6D was better at capturing differences among patients at the top range of the scale with fewer ceiling effects. Both the HUI3 and SF-6D were able to discriminate between patient groups differing in disease severity defined as predialysis versus dialysis dependent and depressive symptoms using a Beck Depression Inventory II score of ≥14 as the cutoff. The HUI3 was better able to discriminate greater depressive symptoms.
Conclusion:  The SF-6D and the HUI3 generate different preference-based scores for patients with CKD and any comparison between their scores should be made with caution. The HUI3 appears more suitable for measuring the health of populations with greater disability such as patients with CKD. It remains to be determined whether these differences will remain when one compares within-instrument differences in preference scores over time.     

Responsiveness of generic health-related quality of life measures in stroke

Objective: To compare five preference-based generic measures of health-related quality of life (HRQOL) in terms of change scores, correlations among change scores, responsiveness, and quality adjusted life-years (QALYs) gained. Design: Observational longitudinal cohort study where clinical measures and self-assessed HRQOL measures were administered to stroke patients at baseline and at 6 months. Patients were categorized as ‘stable’, ‘some improvement’ and ‘large improvement’ using the Barthel Index, Modified Rankin Scale (MRS), and Center for Epidemiologic Studies Depression Scale (CES-D). For each group, paired t -tests and variants of effect size were used to compare the responsiveness of preference-based HRQOL summary scores, including the EQ-5D VAS and index-based score, SF-6D, and Health Utilities Index (HUI) Mark 2 (HUI2) and Mark 3 (HUI3) overall utility scores. Results: Ninety-eight of 124 (79%) patients completed the 6-month follow-up. Change scores of the EQ-Index, HUI2, and HUI3 were strongly correlated with changes in the Barthel Index and MRS, while the EQ-5D VAS had higher correlation with CES-D change scores than the other measures. The SF-6D, HUI3, and EQ-Index were generally more responsive than the HUI2 and EQ-5D Visual analogue scale (EQ-VAS). QALY estimates based on the EQ-5D index and HUI3 were twice as large as estimates based on the SF-6D and HUI2. Conclusions : The results of this study may assist in informing the selection of a preference-based generic HRQOL measure, although choice will also depend on study goals and context. We would caution against the generalization of the study results on responsiveness to conditions when more subtle change is expected.     

Comparing Short Form 6D,Standard Gamble,and Health Utilities Index Mark 2 and Mark 3 utility scores: Results from total hip arthroplasty patients

Objective: The objectives are to compare SF-6D, standard gamble (SG), and Health Utilities Index (HUI) utility scores, compare change scores, and compare responsiveness. Methods: A cohort of osteoarthritis patients referred for total hip arthroplasty (THA) were evaluated at the time of referral and followed until 3months after THA. Patients were assessed using the SF-36, HUI2, HUI3, and the SG. Agreement is assessed using the intra-class correlation (ICC). Responsiveness is assessed using effect size, standardized response mean, and paired t-test. Results: Data was available for 86 patients at baseline and for 63 at both pre- and post-surgery. At baseline mean SF-6D (0.61), SG (0.62), and HUI2 (0.62) scores were similar; the mean HUI3 score (0.52) was lower. Standard deviations were 0.10, 0.32, 0.19, and 0.22. At baseline, agreement between SF-6D and SG scores was 0.13, agreement between SF-6D and HUI2 was 0.47, and agreement between SF-6D and HUI3 was 0.28. Agreement at pre- and post-surgery was similar. The change in scores between post- and pre-surgery was 0.10 for SF-6D, 0.16 for SG, 0.22 for HUI2, and 0.23 for HUI3. Effect sizes were 1.10 for HUI2, 1.08 for HUI3, 1.06 for SF-6D, and 0.48 for the SG. Conclusions: Agreement between SG scores and SF-6D and HUI scores was low. The estimate of change in utility associated with THA was lowest for SF-6D. Additional longitudinal studies to compare utility measures appear to be warranted.     

Comparison of EQ-5D,HUI, and SF-36-derived societal health state values among Spine Patient Outcomes Research Trial (SPORT) participants

Purpose: To compare societal values across health-state classification systems and to describe the performance of these systems at baseline in a large population of persons with confirmed diagnosis of intervertebral disc herniation (IDH), spinal stenosis (SpS), or degenerative spondylolisthesis (DS). Methods: We compared values for EQ-5D (York weights), HUI (Mark 2 and 3), SF-6D, and the SF-36-derived estimate of the Quality of Well Being (eQWB) score using signed rank tests. We tested each instruments ability to discriminate between health categories and level of symptom satisfaction. Correlations were assessed with Spearman rank correlations. We evaluated ceiling and floor effects by comparing the proportion at the highest and the lowest possible score for each tool. In addition, we compared proportions at the highest and lowest levels by dimension. The number of unique health states assigned was compared across instruments. We calculated the difference between those who were very dissatisfied and all others. Results: Mean values ranged from 0.39 to 0.63 among 2097 participants ages 18–93 (mean age 53, 47 female) with significant differences in pair-wise comparisons noted for all systems. Correlations ranged from 0.30 to 0.78. Although all systems showed statistically significant differences in health state values when baseline comparisons were made between those who were very dissatisfied with their symptoms and those who were not, the magnitude of this difference ranged widely across systems. Mean differences (95 CI) between those very dissatisfied and all others were 0.30 (0.269, 0.329) for EQ-5D, 0.22 (0.190, 0.241) for HUI(3), 0.18 (0.161, 0.201) for HUI(2), 0.11 (0.095, 0.117) for SF-6D, 0.04 (0.039, 0.049) for eQWB, and 0.07 (0.056, 0.077) for VAS (with transformation applied to group means). Conclusion: Differences in preference-weighted health state classification systems are evident at baseline in a population with confirmed IDH, SpS, and DS. Caution should be used when comparing health state values derived from various systems.     

Is the Health Utilities Index responsive in total hip arthroplasty patients?

OBJECTIVE: The purpose is to examine the responsiveness of the Health Utilities Index Mark 2 (HUI2), Mark 3 (HUI3), and other generic and disease-specific measures in osteoarthritis patients undergoing total hip arthroplasty (THA). METHODS: Ninety patients (mean age=68.13; SD=8.15) on a waiting list for THA completed measures that included the standard gamble, HUI2, HUI3, SF-36, Harris Hip Scale, WOMAC, and MACTAR. before and after THA. Responsiveness statistics (effect size, standardized response mean, Guyatt's responsiveness statistic, paired-sample t-tests, and relative efficiency statistic) were calculated. RESULTS: The disease-specific measures were more responsive than the generic measures. Rankings of the degree of responsiveness varied depending on the responsiveness statistic used. CONCLUSIONS: Disease-specific measures are the most responsive in THA patients. However, the SF-36, HUI2, and HUI3 had summary scores and domain/attributes scores that were also responsive and provided additional information. Among the generic measures, HUI3 was the most responsive.     

Five preference-based indexes in cataract and heart failure patients were not equally responsive to change

Objective

To compare the responsiveness to clinical change of five widely used preference-based health-related quality-of-life indexes in two longitudinal cohorts.

Study Design and Setting

Five generic instruments were simultaneously administered to 376 adults undergoing cataract surgery and 160 adults in heart failure management programs. Patients were assessed at baseline and reevaluated after 1 and 6 months. The measures were the Short Form (SF)-6D (based on responses scored from SF-36v2), Self-Administered Quality of Well-being Scale (QWB-SA), the EuroQol-5D developed by the EuroQol Group, the Health Utilities Indexes Mark 2 (HUI2) and Mark 3 (HUI3). Cataract patients completed the National Eye Institute Visual Functioning Questionnaire-25, and heart failure patients completed the Minnesota Living with Heart Failure Questionnaire. Responsiveness was estimated by the standardized response mean.

Results

For cataract patients, mean changes between baseline and 1-month follow-up for the generic indices ranged from 0.00 (SF-6D) to 0.052 (HUI3) and were statistically significant for all indexes except the SF-6D. For heart failure patients, only the SF-6D showed significant change from baseline to 1 month, whereas only the QWB-SA change was significant between 1 and 6 months.

Conclusions

Preference-based methods for measuring health outcomes are not equally responsive to change.     

Assessing the validity of the SF-36 General Health Survey

Our objective was to assess the validity of the SF-36 General Health Survey against the Social Maladjustment Schedule (SMS) and two questionnaire measures, the Social Problem Questionnaire and the Nottingham Health Profile (NHP) in a random subsample of 206 men and women from the Whitehall II study, a longitudinal survey of health and disease amongst 10,308 London-based civil servants. We found that social functioning on the SF-36 correlated significantly with social contacts, total satisfaction and total management scores on the SMS, and social isolation and emotional reactions on the NHP. General mental health on the SF-36 was associated with marriage, social contacts, leisure scores, total satisfaction and total management scores on the SMS, and emotional reactions, energy level and social isolation on the NHP. Conversely, physical functioning and physical role limitations were generally not associated with the SMS but were associated with physical abilities and pain on the NHP. In conclusion, this study offers evidence of the discriminant validity of the general mental health and physical functioning scales of the SF-36. We also found moderate construct and criterion validity for the social functioning scale of the SF-36 and considerable overlap between the general mental health and social functioning scales.     

A Study of the Construct Validity of the Health Utilities Index Mark 3 (HUI3) in Patients with Schizophrenia

This study assessed the construct validity of the Health Utilities Index Mark 3 (HUI3) in patients with schizophrenia. Patients with schizophrenia recruited from a tertiary mental hospital in Singapore completed the HUI3, the Short-Form 36 Health Survey (SF-36) and the Schizophrenia Quality of Life Scale (SQLS). Patients were assessed for presence and absence of 22 common psychiatric symptoms. Construct validity was assessed using 6 a priori hypotheses. Two hundred and two patients (mean age: 37.8 years, female: 52%) completed the survey. As hypothesized, overall HUI3 utility scores were correlated with SF-36 measures (Spearman’s rho: 0.19 to 0.51), SQLS scales (Spearman’s rho: −0.56 to −0.36), and the number of psychiatric symptoms (Spearman’s rho: −0.49). The HUI3 emotion attribute was moderately correlated with SF-36 mental health (Spearman’s rho: 0.45) and SQLS psychosocial scales (Spearman’s rho: −0.43), and HUI3 pain attribute was strongly correlated with SF-36 bodily pain scale (Spearman’s rho: 0.58). The mean HUI3 overall, emotion, cognition, and speech scores for patients with schizophrenia were 0.07, 0.09, 0.04 and 0.04 points lower than respective age-, sex- and ethnicity-adjusted population norms (p<0.001 for all, ANCOVA). This study provides evidence for the construct validity of the HUI3 in patients with schizophrenia.     

Reliability, validity, and minimally important differences of the SF-6D in systemic sclerosis

OBJECTIVES: To evaluate the reliability and validity and estimate the minimally important difference (MID) for the SF-6D in patients with systemic sclerosis (SSc). Subjects We used data from two clinical studies to analyze the SF-6D in patients with SSc: Study 1 was a cross-sectional observational study (N = 107) designed to assess three direct preference measures--the rating scale, time trade-off, and standard gamble (SG) in patients with diffuse SSc and limited SSc, and Study 2 was a 12-month randomized, placebo-controlled, clinical trial (N = 168) assessing oral bovine collagen versus placebo in diffuse SSc. METHODS: We assessed the test-retest reliability of the SF-6D in Study 2 over a mean (SD) 4.8 (3.0)-week interval and the agreement between the SF-6D and direct preference measures in Study 1 using intraclass correlations (ICC). The MID was estimated using three different anchors--the SF-36 change in health item (patients who answered "somewhat better" formed the MID group), the Health Assessment Questionnaire-Disability Index (HAQ-DI; change of > or =0.14 and > or =0.22) and the skin score (change of > or =5.3). Results The mean (SD) SF-6D scores were 0.61 (0.12) in Study 1 and 0.64 (0.13) in Study 2. Test-retest reliability for the SF-6D was high (ICC = 0.82 [95% CI: 0.76, 0.87]). Agreement between the SF-6D and three direct preferences measures was poor to moderate (0.16-0.52). The MID estimate for the SF-6D using the change in SF-36 item -0.012 and this level of change was similar to the no change group. The mean MID estimate for the SF-6D improvement using the HAQ-DI and skin score as anchors was 0.035 (effect size of 0.27). CONCLUSION: This is the first study to assess the SF-6D in SSc. The SF-6D is reliable and valid in patients with SSc. We provide MID estimates that can aid in calculating sample size for clinical trials involving patients with diffuse SSc.     

An assessment of validity and responsiveness of generic measures of health-related quality of life in hearing impairment

Purpose

This review examines psychometric performance of three widely used generic preference-based measures, that is, EuroQol 5 dimensions (EQ-5D), Health Utility Index 3 (HUI3) and Short-form 6 dimensions (SF-6D) in patients with hearing impairments.

Methods

A systematic search was undertaken to identify studies of patients with hearing impairments where health state utility values were measured and reported. Data were extracted and analysed to assess the reliability, validity (known group differences and convergent validity) and responsiveness of the measures across hearing impairments.

Results

Fourteen studies (18 papers) were included in the review. HUI3 was the most commonly used utility measures in hearing impairment. In all six studies, the HUI3 detected difference between groups defined by the severity of impairment, and four out of five studies detected statistically significant changes as a result of intervention. The only study available suggested that EQ-5D only had weak ability to discriminate difference between severity groups, and in four out of five studies, EQ-5D failed to detected changes. Only one study involved the SF-6D; thus, the information is too limited to conclude on its performance. Also evidence for the reliability of these measures was not found.

Conclusion

Overall, the validity and responsiveness of the HUI3 in hearing impairment was good. The responsiveness of EQ-5D was relatively poor and weak validity was suggested by limited evidence. The evidence on SF-6D was too limited to make any judgment. More head-to-head comparisons of these and other preference measures of health are required.     

The feasibility and responsiveness of the health utilities index in patients with early-stage breast cancer: A prospective longitudinal study

Background The health utilities index (HUI3) is a health measurement instrument based on individuals’ preferences for different health states. Breast cancer (BC) is common, with a high proportion of long-term survivors, making evaluation of treatment effects important. Feasibility and responsiveness of HUI3 was compared to the short-form 36 (SF-36) in patients with BC. Methods HUI3 and SF-36 were administered eight times: at initial surgical consultation, 1 week before surgery; 1 week, 3, 6, 12, 18, and 24 months after surgery. Effect size, analysis of variance, and Pearson product moment correlations were calculated. BC data were compared to normative values. Results Eighty-five patients were enrolled. Ninety-one percent of planned assessments were completed. HUI3 showed significant responsiveness (P < 0.01) after surgery and during recovery. HUI3 scores correlated with SF-36 scores. Comparison to normative data demonstrated the significant detrimental effect of BC diagnosis. Results showed long-term effects of treatment on physical health and positive effects on mental/emotional health in BC survivors. Conclusion(s) HUI3 was found to be feasible and responsive in our cohort of BC patients. Changes in HUI3 values over time, and compared to normative data, paralleled SF-36 scores. HUI3 is a valuable tool in health-related quality of life and cost-utility studies in patients with BC.     

Discordance in Utility Measurement in Persons with Neurological Conditions: A Comparison of the SF-6D and the HUI3

Objectives

To examine the extent of disagreement in estimated utility between the six-dimensional health state short form (SF-6D) and the Health Utilities Index—Mark 3 (HUI3) in Canadians with neurological conditions and how discordance varied by participant and neurological condition attributes.

Are indirect utility measures reliable and responsive in rheumatoid arthritis patients?

Background: Preference-based, generic measures are increasingly being used to measure quality of life and as sources for quality weights in the estimation of Quality Adjusted Life Years (QALYs) in rheumatoid arthritis (RA). However, among the most commonly used instruments (the Health Utilities Index 2 and 3 [HUI2 and HUI3], the EuroQoL-5D [EQ-5D], and the Short Form-6D [SF-6D], there has been little comparative research. Therefore, we examined the reliability and responsiveness of these measures and the Rheumatoid Arthritis Quality of Life (RAQoL) and the Health Assessment Questionnaire (HAQ) in a sample of RA patients. Major findings: Test–retest reliability was acceptable for all of the instruments with the exception of the EQ-5D. Using two external criteria to define change (a patient transition question and categories of the patient global assessment of disease activity VAS), the RAQoL was the most responsive of the instruments. For the indirect utility instruments, the HUI3 and the SF-6D were the most responsive for measuring positive change. On average, for patients whose RA improved, the absolute change was highest for the HUI3. Conclusions: The HUI3 and the SF-6D appear to be the most responsive of the preference-based instruments in RA. However, differences in the magnitude of the absolute change scores have important implications for cost-effectiveness analyses.     

A comparison of generic, indirect utility measures (the HUI2, HUI3, SF-6D, and the EQ-5D) and disease-specific instruments (the RAQoL and the HAQ) in rheumatoid arthritis

Rheumatoid arthritis (RA) is a common, chronic disease where health-related quality of life (HRQL) is one of the main goals of therapy. As such, instruments used to measure HRQL in RA must be able to discriminate across RA severity. The two basic categories of instruments used to measure HRQL are generic instruments and disease-specific instruments. Generic instruments can be further subdivided into preference-based measures which yield both single and multi-attribute utility values anchored at zero (death) and 1.00 (perfect health) as a measure of HRQL. The scores from these types of instruments can be integrated into cost-utility analyses as the weightings for quality adjusted life years. We assessed the construct validity of utility scores from four generic preference-based measures (the Health Utilities Index 2 and 3 (HUI2, HUI3), the EuroQol 5D (EQ-5D), and the Short Form 6-D (SF-6D) and disease specific measures (the Rheumatoid Arthritis Quality of Life Questionnaire (RAQoL) and the Health Assessment Questionnaire (HAQ)) in a sample of 313 RA patients in British Columbia, Canada. We also estimated the minimally important differences (MID) for each of the measures. Generally, as anticipated, the disease-specific measures were better able to discriminate across groups with higher RA severity; however, utility scores from each of the scales also appeared to discriminate well across RA severity categories. The MID values agreed with those previously reported in the literature for the HUI2, SF-6D and the HAQ and provided new information for the HUI3, EQ-5D and the RAQoL. We conclude that the all of the preference-based utility measures that were evaluated appear to adequately discriminate across levels of RA severity.