Item response theory was used to shorten EORTC QLQ-C30 scales for use in palliative care

BACKGROUND AND OBJECTIVE: The goal was to develop a shortened version of the EORTC QLQ-C30 for use in palliative care. We wanted to keep as few items as possible in each scale while still being able to compare results with studies using the original scales. We examined the possibilities of shortening the physical functioning, cognitive functioning, fatigue, and nausea and vomiting scales. STUDY DESIGN AND SETTING: The shortening was based on 2,366 (physical functioning) and 10,815 (three other scales) observations, respectively. We used item response theory to construct scoring algorithms for predicting scores on the original scales. RESULTS: Evaluations showed that a three-item physical scale, a two-item fatigue scale, and a one-item nausea or vomiting scale predicted the scores on the original scales with excellent agreement and had measurement abilities similar to the original scales with no loss or only a little loss in power to detect group differences. The results of the cognitive functioning scale indicated problems when predicting scores from a shortened version. CONCLUSION: Given the favorable results for the physical functioning, fatigue, and nausea or vomiting scales we expect that the shortened versions of these scales will be included in the abbreviated version of the EORTC QLQ-C30 for palliative care.     

Development of computerized adaptive testing (CAT) for the EORTC QLQ-C30 physical functioning dimension

Purpose

Computerized adaptive test (CAT) methods, based on item response theory (IRT), enable a patient-reported outcome instrument to be adapted to the individual patient while maintaining direct comparability of scores. The EORTC Quality of Life Group is developing a CAT version of the widely used EORTC QLQ-C30. We present the development and psychometric validation of the item pool for the first of the scales, physical functioning (PF).

Methods

Initial developments (including literature search and patient and expert evaluations) resulted in 56 candidate items. Responses to these items were collected from 1,176 patients with cancer from Denmark, France, Germany, Italy, Taiwan, and the United Kingdom. The items were evaluated with regard to psychometric properties.

Results

Evaluations showed that 31 of the items could be included in a unidimensional IRT model with acceptable fit and good content coverage, although the pool may lack items at the upper extreme (good PF). There were several findings of significant differential item functioning (DIF). However, the DIF findings appeared to have little impact on the PF estimation.

Conclusions

We have established an item pool for CAT measurement of PF and believe that this CAT instrument will clearly improve the EORTC measurement of PF.     

Use of item response theory to develop a shortened version of the EORTC QLQ-C30 emotional functioning scale

Background: As part of a larger study whose objective is to develop an abbreviated version of the EORTC QLQ-C30 suitable for research in palliative care, analyses were conducted to determine the feasibility of generating a shorter version of the 4-item emotional functioning (EF) scale that could be scored in the original metric. Methods: We used data from 24 European cancer studies conducted in 10 different languages (n=8242). Item selection was based on analyses by item response theory (IRT). Based on the IRT results, a simple scoring algorithm was developed to predict the original 4-item EF sum scale score from a reduced number of items. Results: Both a 3-item and a 2-item version (item 21 ‘Did you feel tense?’ and item 24 ‘Did you feel depressed?’) predicted the total score with excellent agreement and very little bias. In group comparisons, the 2-item scale led to the same conclusions as those based on the original 4-item scale with little or no loss of measurement efficiency. Conclusion: Although these results are promising, confirmatory studies are needed based on independent samples. If such additional studies yield comparable results, incorporation of the 2-item EF scale in an abbreviated version of the QLQ-C30 for use in palliative care research settings would be justified. The analyses reported here demonstrate the usefulness of the IRT-based methodology for shortening questionnaire scales.     

The EORTC computer-adaptive tests measuring physical functioning and fatigue exhibited high levels of measurement precision and efficiency

ObjectivesThe European Organisation for Research and Treatment of Cancer (EORTC) Quality of Life Group is developing a computer-adaptive test (CAT) version of the EORTC Quality of Life Questionnaire (QLQ-C30). We evaluated the measurement properties of the CAT versions of physical functioning (PF) and fatigue (FA) and compared these with the corresponding QLQ-C30 scales.Study Design and SettingBased on international samples of more than 1,000 cancer patients, we simulated CAT administration of varying numbers of items and compared the resulting scores with those based on all items in the respective item pools. Furthermore, the relative validity (RV) of CATs was compared with that of the QLQ-C30 scales using known groups validity.ResultsFor both dimensions, CATs of all lengths resulted in unbiased score estimates. CATs consisting of five or more items had reliability>0.90, correlated ≥0.97 with the full scale, and had root mean square error <0.25. The average RVs for these CATs ranged 1.02–1.33, indicating possible savings in sample size requirements of 3–42% using CAT.ConclusionThe CAT versions of PF and FA exhibited high levels of measurement precision and efficiency. The potential savings in sample size requirements using CATs compared with those using the original QLQ-C30 scales were typically 20% or more.     

Psychometric evaluation of the EORTC computerized adaptive test (CAT) fatigue item pool

Purpose

Fatigue is one of the most common symptoms associated with cancer and its treatment. To obtain a more precise and flexible measure of fatigue, the EORTC Quality of Life Group has developed a computerized adaptive test (CAT) measure of fatigue. This is part of an ongoing project developing a CAT version of the widely used EORTC QLQ-C30 questionnaire.

Methods

Based on the literature search and evaluations by experts and patients, 41 new fatigue items were developed (in addition to the three QLQ-C30 fatigue items). Psychometric properties of the items, including evaluations of dimensionality, fit to item response theory (IRT) model, and differential item functioning (DIF), were assessed in an international sample of cancer patients.

Results

Responses were obtained from 1,321 cancer patients coming from eight countries. Factor analysis showed that 37 of the items could be included in a unidimensional model (RMSEA = 0.098, TLI = 0.995, CFI = 0.920). Of the 37 items, two were deleted because of poor fit to the IRT model forming the basis for the CAT, and one because of DIF between cancer sites.

Conclusions

We have established a 34-item fatigue bank allowing for more precise and flexible measurement of fatigue, while still being backward compatible with the QLQ-C30 fatigue scale.     

Multidimensional Computerized Adaptive Testing of the EORTC QLQ-C30: Basic Developments and Evaluations

Objective: Self-report questionnaires are widely used to measure health-related quality of life (HRQOL). Ideally, such questionnaires should be adapted to the individual patient and at the same time scores should be directly comparable across patients. This may be achieved using computerized adaptive testing (CAT). Usually, CAT is carried out for a single domain at a time. However, many HRQOL domains are highly correlated. Multidimensional CAT may utilize these correlations to improve measurement efficiency. We investigated the possible advantages and difficulties of multidimensional CAT. Study design and setting: We evaluated multidimensional CAT of three scales from the EORTC QLQ-C30: the physical functioning, emotional functioning, and fatigue scales. Analyses utilised a database with 2958 European cancer patients.Results: It was possible to obtain scores for the three domains with five to seven items administered using multidimensional CAT that were very close to the scores obtained using all 12 items and with no or little loss of measurement precision.Conclusion: The findings suggest that multidimensional CAT may significantly improve measurement precision and efficiency and encourage further research into multidimensional CAT. Particularly, the estimation of the model underlying the multidimensional CAT and the conceptual aspects need further investigations.     

Psychometric evaluation of an item bank for computerized adaptive testing of the EORTC QLQ-C30 cognitive functioning dimension in cancer patients

Background

The European Organisation of Research and Treatment of Cancer (EORTC) Quality of Life Group is developing computerized adaptive testing (CAT) versions of all EORTC Quality of Life Questionnaire (QLQ-C30) scales with the aim to enhance measurement precision. Here we present the results on the field-testing and psychometric evaluation of the item bank for cognitive functioning (CF).

Methods

In previous phases (I–III), 44 candidate items were developed measuring CF in cancer patients. In phase IV, these items were psychometrically evaluated in a large sample of international cancer patients. This evaluation included an assessment of dimensionality, fit to the item response theory (IRT) model, differential item functioning (DIF), and measurement properties.

Results

A total of 1030 cancer patients completed the 44 candidate items on CF. Of these, 34 items could be included in a unidimensional IRT model, showing an acceptable fit. Although several items showed DIF, these had a negligible impact on CF estimation. Measurement precision of the item bank was much higher than the two original QLQ-C30 CF items alone, across the whole continuum. Moreover, CAT measurement may on average reduce study sample sizes with about 35–40% compared to the original QLQ-C30 CF scale, without loss of power.

Conclusion

A CF item bank for CAT measurement consisting of 34 items was established, applicable to various cancer patients across countries. This CAT measurement system will facilitate precise and efficient assessment of HRQOL of cancer patients, without loss of comparability of results.

     

Testing psychometric properties of the standard Chinese version of the European Organization for Research and Treatment of Cancer Quality of Life Core Questionnaire 30 (EORTC QLQ-C30)

BACKGROUND: Because cancer is now the first and second leading causes of death in both of urban and suburban area in China, there are increasing demands for measurement tools to assess quality of life in Chinese cancer patients. The objective of this study was to evaluate the psychometric properties of the standard Chinese version of the European Organization for Research and Treatment of Cancer Quality of Life Core Questionnaire 30 (EORTC QLQ-C30, version 3.0). METHODS: The questionnaire was administered before, during, and after treatment of 143 patients with breast, gynecological, or lung cancer in six hospitals in China. RESULTS: Cronbach's alpha coefficients for multi-item scales were greater than 0.70 before and during treatment, except for the cognitive functioning scale. Multitrait scaling analysis showed that most of the item-scale correlation coefficients met the standards of convergent and discriminant validity. All scales and items exhibited good reproducibility, criterion-related validity, and construct validity. Score changes over time were observed in the following scales: physical, role, and social functioning; global quality of life; fatigue; nausea/vomiting. Score changes were also observed in the appetite loss item. CONCLUSION: The standard Chinese version of the EORTC QLQ-C30 is overall a valid instrument to assess the quality of life of Chinese cancer patients.     

The rand 36-item health survey 1.0

Recently, Ware and Sherbourne¹ published a new short-form health survey, the MOS 36-Item Short-Form Health Survey (SF-36), consisting of 36 items included in long-form measures developed for the Medical Outcomes Study. The SF-36 taps eight health concepts: physical functioning, bodily pain, role limitations due to physical health problems, role limitations due to personal or emotional problems, general mental health, social functioning, energy/fatigue, and general health perceptions. It also includes a single item that provides an indication of perceived change in health. The SF-36 items and scoring rules are distributed by MOS Trust, Inc. Strict adherence to item wording and scoring recommendations is required in order to use the SF-36 trademark. The RAND 36-Item Health Survey 1.0 (distributed by RAND) includes the same items as those in the SF-36, but the recommended scoring algorithm is somewhat different from that of the SF-36. Scoring differences are discussed here and new T-scores are presented for the 8 multi-item scales and two factor analytically-derived physical and mental health composite scores.     

Testing the measurement equivalence of paper and interactive voice response system versions of the EORTC QLQ-C30

Purpose

The objective of this study was to evaluate the measurement equivalence of an interactive voice response system (IVRS) version and the original paper-based version of the EORTC QLQ-C30.

Methods

The QLQ-C30 is a cancer-specific, health-related quality of life questionnaire consisting of nine multi-item scales (physical, role, emotional, cognitive and social functioning, fatigue, nausea/vomiting, pain, and quality of life) and six single item measures (dyspnea, insomnia, appetite loss, constipation, diarrhea, and financial problems). This study utilized a crossover design with subjects randomly assigned to one of two assessment orders: (1) paper then IVRS or (2) IVRS then paper. Equivalence between the two administration modes was established by comparing the 95 % lower confidence interval (CI) of the intraclass correlation coefficients (ICCs) for each scale, with a critical value of 0.70.

Results

The ICCs for the nine multi-item scales were all above 0.79, ranging from 0.791 to 0.899 (ICC 95 % lower CI range 0.726–0.865) and significantly different from our threshold reliability of 0.70. The ICCs for the six single items ranged from 0.689 to 0.896 (ICC 95 % lower CI range 0.611–0.888). Two of the items, insomnia and appetite loss, were not statistically different from 0.70. When considered together, the per-protocol analysis results support the equivalence of the paper and IVRS versions of the QLQ-C30 for 13 of the 15 scores.

Conclusion

This analysis provides evidence that the scores obtained from the IVRS version of the QLQ-C30 are equivalent to those obtained with the original paper version except for the insomnia and appetite loss items.     

Quality of life scores differed according to mode of administration in a review of three major oncology questionnaires

OBJECTIVE: To assess whether scores on the three major quality-of-life questionnaires in oncology (FACT-G, FLIC, and EORTC QLQ-C30) are associated with modes of administration in a realistic clinical research setting. STUDY DESIGN AND SETTING: A heterogeneous sample of 1,265 cancer patients was recruited in Singapore. About one-fourth of the patients chose to have the interview administered by research staff; the rest self-completed the questionnaires. Multiple regression was used to adjust for differences in demographic and clinical characteristics between patients. An equivalence margin was defined as 0.25 standard deviations. RESULTS: Apart from one exception (the EORTC QLQ-C30 global functioning scale), all scales showed higher mean values in patients who were interviewed than patients who self-administered the questionnaires. For the physical and functional well-being scales of FACT-G and the physical and social functioning scales of EORTC QLQ-C30, the differences were small and the confidence intervals fell totally within the equivalence zone. The emotional well-being score of the FACT-G was different across modes of administration and the confidence interval fell outside the equivalence zones. There was no interaction between modes of administration and respondents' education level. CONCLUSION: The physical aspect of quality-of-life is not sensitive to interviewer administration but the psychological aspect is. Statistical adjustment for some scales is recommended.     

Quality of life of lung cancer patients: Validation of the Taiwan Chinese version of the EORTC QLQ-C30 and QLQ-LC13

The purpose of this study was to test the validity and reliability of the Taiwan Chinese translation of the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire Core-30 (EORTC QLQ-C30, version 3) and Quality of Life Questionnaire Lung Cancer-13 (QLQ-LC13) questionnaires. Consecutively 51 patients with lung cancer undergoing active chemotherapy and 48 such patients undergoing regular follow-up completed the questionnaires. The intraclass correlation between test and retest ranged from 0.46 to 0.85 for the QLQ-C30 and was 0.76 for dyspnea for the QLQ-LC13. The kappa coefficients between test and retest ranged from 0.51 to 0.73 for single items of the QLQ-C30 and 0.49-0.68 for five of the nine items in the QLQ-LC13. The Cronbach's alpha coefficients were > or = 0.70 for all scales of the two questionnaires apart from that of cognitive functioning. The correlation coefficients between indices measuring similar dimensions of the EORTC QLQ-C30 and the SF-36 questionnaires ranged from 0.43 to 0.73, and that between the dyspnea scales of the two EORTC questionnaires was 0.70. Patients in the follow-up group revealed higher scores of global status/quality of life, and lower scores of nausea/vomiting, as also physical functioning. The questionnaires could also detect expected adverse effects of radiotherapy, cisplatin, and paclitaxel.     

Assessing the reliability of the EORTC QLQ-C30 in a sample of older African American and Caucasian adults

The purpose of this study was to examine the structure and reliability of the EORTC QLQ-C30. This 30-item instrument has five functional scales (physical, role, cognitive, emotional, and social), three symptom scales (fatigue, pain, and nausea and vomiting) and a global health and quality of life scale. Confirmatory factor analysis and Cronbach's α estimates were used to assess the functioning of the EORTC QLQ-C30 in a sample of 489 African American (n = 255) and Caucasian (n = 234) adults aged 50 + years. Seven of the nine EORTC QLQ-C30 scales showed good reliability for both the African Americans and the Caucasians in the sample (Cronbach's > 0.75). In contrast, the cognitive functioning scale had a reliability coefficient of only 0.69 for the African Americans and 0.40 for the Caucasians, and the nausea and vomiting scale had a reliability coefficient of only 0.49 for the African Americans and 0.51 for the Caucasians. In summary, although the overall reliabilities of seven of the scales showed good fit, many of the item-to-scale correlations did not. Researchers planning to use the EORTC QLQ-C30 might first consider conducting separate analyses on the different racial or ethnic subgroups in their study populations to determine whether a common set of factors or scales is available for further analysis. This revised version was published online in June 2006 with corrections to the Cover Date.     

QLU-C10D: a health state classification system for a multi-attribute utility measure based on the EORTC QLQ-C30

Purpose

To derive a health state classification system (HSCS) from the cancer-specific quality of life questionnaire, the EORTC QLQ-C30, as the basis for a multi-attribute utility instrument.

Methods

The conceptual model for the HSCS was based on the established domain structure of the QLQ-C30. Several criteria were considered to select a subset of dimensions and items for the HSCS. Expert opinion and patient input informed a priori selection of key dimensions. Psychometric criteria were assessed via secondary analysis of a pooled dataset comprising HRQOL and clinical data from 2616 patients from eight countries and a range of primary cancer sites, disease stages, and treatments. We used confirmatory factor analysis (CFA) to assess the conceptual model’s robustness and generalisability. We assessed item floor effects (>75 % observations at lowest score), disordered item response thresholds, coverage of the latent variable and differential item function using Rasch analysis. We calculated effect sizes for known group comparisons based on disease stage and responsiveness to change. Seventy-nine cancer patients assessed the relative importance of items within dimensions.

Results

CFA supported the conceptual model and its generalisability across primary cancer sites. After considering all criteria, 12 items were selected representing 10 dimensions: physical functioning (mobility), role functioning, social functioning, emotional functioning, pain, fatigue, sleep, appetite, nausea, bowel problems.

Conclusions

The HSCS created from QLQ-C30 items is known as the EORTC Quality of Life Utility Measure-Core 10 dimensions (QLU-C10D). The next phase of the QLU-C10D’s development involves valuation studies, currently planned or being conducted across the globe.

     

Item response theory facilitated cocalibrating cognitive tests and reduced bias in estimated rates of decline

OBJECTIVE: To cocalibrate the Mini-Mental State Examination, the Modified Mini-Mental State, the Cognitive Abilities Screening Instrument, and the Community Screening Instrument for Dementia using item response theory (IRT) to compare screening cut points used to identify cases of dementia from different studies, to compare measurement properties of the tests, and to explore the implications of these measurement properties on longitudinal studies of cognitive functioning over time. STUDY DESIGN AND SETTING: We used cross-sectional data from three large (n>1000) community-based studies of cognitive functioning in the elderly. We used IRT to cocalibrate the scales and performed simulations of longitudinal studies. RESULTS: Screening cut points varied quite widely across studies. The four tests have curvilinear scaling and varied levels of measurement precision, with more measurement error at higher levels of cognitive functioning. In longitudinal simulations, IRT scores always performed better than standard scoring, whereas a strategy to account for varying measurement precision had mixed results. CONCLUSION: Cocalibration allows direct comparison of cognitive functioning in studies using any of these four tests. Standard scoring appears to be a poor choice for analysis of longitudinal cognitive testing data. More research is needed into the implications of varying levels of measurement precision.     

Comparing translations of the EORTC QLQ-C30 using differential item functioning analyses

The European Organisation for Research and Treatment of Cancer (EORTC) QLQ-C30 is one of the most widely used quality of life instruments for cancer patients. The aim of this study was to assess whether there were linguistic differences in the way an international sample answered the EORTC QLQ-C30 questionnaire. Thirteen translations of the EORTC QLQ-C30, representing 22 countries, were investigated using a database of 27,891 respondents, incorporating 103 separate studies. Differential item functioning (DIF) analyses were conducted using logistic regression to identify items which, after controlling for subscale, were answered differently by language of administration. Both uniform and non-uniform DIF were assessed. Although most languages showed similar results to English, at least one instance of statistically significant DIF was identified for each translation, and a few of these differences were large. In some cases, the patterns were supported by the results of qualitative interviews with bilingual people. Although, overall, there appeared to be good linguistic equivalence for most of the EORTC QLQ-C30 items, several scales showed strongly discrepant results for some translations. Some of these effects are large enough to impact on the results of clinical trials. Based on our experience in this study, we suggest that validation of translations of health-related quality of life instruments should include exploration of DIF.     

A validation study of the domains of the core EORTC Quality of Life Questionnaire

A cohort study was conducted to investigate the construct validity of the domains of the EORTC QLQ-C30 (European Organization for the Research and Treatment of Cancer Quality of Life Questionnaire). Ninety-six patients undergoing chemotherapy at the Kingston Regional Cancer Centre, Kingston, Canada were given the EORTC QLQ-C30 and two out of four comparison scales; the Sickness Impact Profile, the McGill Pain Questionnaire, the General Health Questionnaire and the Cancer Rehabilitation Evaluation System, to complete during their clinic visits. Using the multitrait-multimethod matrix to examine relationships, the Spearman ranked correlation coefficients of similar and dissimilar domains between the EORTC QLQ-C30 and the comparison scales were compared. The EORTC QLQ-C30 domains of pain and physical and role functioning performed well in that they converged and diverged appropriately with the other instruments. The domain of psychological functioning had strong associations with domains thought to be dissimilar (social interaction and fatigue). Social interaction and financial impact diverged across dissimilar domains. This lack of specificity may relate to question wording. The symptom domain had strong associations with other domains across the comparison scales. This study shows that the domains of the EORTC QLQ-C30 have substantial construct validity in the multidimensional assessment of the quality of life of cancer patients and identifies where further work is required.     

The OVIS study: health related quality of life measured by the EORTC QLQ-C30 and -BR23 in German female patients with breast cancer from Schleswig-Holstein

The OVIS study is a population-based study that aims at evaluating medical care in Schleswig-Holstein (Germany). In this paper, the health related quality of life (QoL, EORTC QLQ-C30 and -BR23) of 1,927 women with breast cancer is reported. The global health status/QoL score was comparable to reference data of the age-adjusted German general population, but clinical meaningful differences (≥ 10 points) were found for all functioning scales (with the exception of physical functioning) and for three of the symptom scales/items (fatigue, dyspnoe, insomnia) with OVIS patients showing more deficits. Furthermore, OVIS patients scored higher on the item financial difficulties. Logistic regression analyses revealed that coming from an urban surrounding, having a higher social status and attendance to a regular aftercare predicted a good overall QoL, while factors that were related to perceived complications in the course of the therapy raised the risk for a low QoL rating. It is of interest, that attendance to a rehabilitation and interest in self-help groups independently predicted an increased risk for a low quality of life. Overall, we assume the global QoL is rating slightly too optimistic since major deficits were reported on the specific physical and functional scales/items.     

The feasibility of applying item response theory to measures of migraine impact: A re-analysis of three clinical studies

Background: Item response theory (IRT) is a powerful framework for analyzing multiitem scales and is central to the implementation of computerized adaptive testing. Objectives: To explain the use of IRT to examine measurement properties and to apply IRT to a questionnaire for measuring migraine impact – the Migraine Specific Questionnaire (MSQ). Methods: Data from three clinical studies that employed the MSQ-version 1 were analyzed by confirmatory factor analysis for categorical data and by IRT modeling. Results: Confirmatory factor analyses showed very high correlations between the factors hypothesized by the original test constructions. Further, high item loadings on one common factor suggest that migraine impact may be adequately assessed by only one score. IRT analyses of the MSQ were feasible and provided several suggestions as to how to improve the items and in particular the response choices. Out of 15 items, 13 showed adequate fit to the IRT model. In general, IRT scores were strongly associated with the scores proposed by the original test developers and with the total item sum score. Analysis of response consistency showed that more than 90% of the patients answered consistently according to a unidimensional IRT model. For the remaining patients, scores on the dimension of emotional function were less strongly related to the overall IRT scores that mainly reflected role limitations. Such response patterns can be detected easily using response consistency indices. Analysis of test precision across score levels revealed that the MSQ was most precise at one standard deviation worse than the mean impact level for migraine patients that are not in treatment. Thus, gains in test precision can be achieved by developing items aimed at less severe levels of migraine impact. Conclusions: IRT proved useful for analyzing the MSQ. The approach warrants further testing in a more comprehensive item pool for headache impact that would enable computerized adaptive testing.