The development of a new computer adaptive test to evaluate chorea in Huntington disease: HDQLIFE Chorea

Purpose

Huntington’s disease (HD) is an autosomal dominant neurodegenerative disease associated with motor, behavioral, and cognitive deficits. The hallmark symptom of HD, chorea, is often the focus of HD clinical trials. Unfortunately, there are no self-reported measures of chorea. To address this shortcoming, we developed a new measure of chorea for use in HD, HDQLIFE Chorea.

Methods

Qualitative data and literature reviews were conducted to develop an initial item pool of 141 chorea items. An iterative process, including cognitive interviews, expert review, translatability review, and literacy review, was used to refine this item pool to 64 items. These 64 items were field tested in 507 individuals with prodromal and/or manifest HD. Exploratory and confirmatory factor analyses (EFA and CFA, respectively) were conducted to identify a unidimensional set of items. Then, an item response theory graded response model (GRM) and differential item functioning analyses were conducted to select the final items for inclusion in this measure.

Results

EFA and CFA supported the retention of 34 chorea items. GRM and DIF supported the retention of all of these items in the final measure. GRM calibration data were used to inform the selection of a 6-item, static short form and to program the HDQLIFE Chorea computer adaptive test (CAT). CAT simulation analyses indicated a 0.99 correlation between the CAT scores and the full item bank.

Conclusions

The new HDQLIFE Chorea CAT and corresponding 6-item short form were developed using established rigorous measurement development standards; this is the first self-reported measure developed to evaluate the impact of chorea on HRQOL in HD. This development work indicates that these measures have strong psychometric properties; future work is needed to establish test–retest reliability and responsiveness to change.

     

HDQLIFE: the development of two new computer adaptive tests for use in Huntington disease,Speech Difficulties,and Swallowing Difficulties

Purpose

Huntington disease (HD) is an autosomal dominant neurodegenerative disease which results in several progressive symptoms, including bulbar dysfunction (i.e., speech and swallowing difficulties). Although difficulties in speech and swallowing in HD have a negative impact on health-related quality of life, no patient-reported outcome measure exists to capture these difficulties that are specific to HD. Thus, we developed a new patient-reported outcome measure for use in the Huntington Disease Health-Related Quality of Life (HDQLIFE) Measurement System that focused on the impact that difficulties with speech and swallowing have on HRQOL in HD.

Methods

Five hundred and seven individuals with prodromal and/or manifest HD completed 47 newly developed items examining speech and swallowing difficulties. Unidimensional item pools were identified using exploratory factor analysis and confirmatory factor analysis (EFA and CFA, respectively). Item response theory (IRT) was used to calibrate the final measures.

Results

EFA and CFA identified two separate unidimensional sets of items: Speech Difficulties (27 items) and Swallowing Difficulties (16 items). Items were calibrated separately for these two measures and resulted in item banks that can be administered as computer adaptive tests (CATs) and/or 6-item, static short forms. Reliability of both of these measures was supported through high correlations between the simulated CAT scores and the full item bank.

Conclusions

CATs and 6-item calibrated short forms were developed for HDQLIFE Speech Difficulties and HDQLIFE Swallowing Difficulties. These measures both demonstrate excellent psychometric properties and may have clinical utility in other populations where speech and swallowing difficulties are prevalent.

     

Understanding patient-reported outcome measures in Huntington disease: at what point is cognitive impairment related to poor measurement reliability?

Purpose

Symptom progression in Huntington disease (HD) is associated with cognitive decline which may interfere with the self-report of symptoms. Unfortunately, data to support or refute the psychometric reliability of patient-reported outcomes (PROs) as HD progresses are limited. This is problematic given that PROs are increasingly recognized as important measures of efficacy for new treatments.

Methods

We examined PRO data from the HDQLIFE Measurement System (Speech Difficulties; Swallowing Difficulties; Chorea) in 509 individuals with premanifest, early-stage, or late-stage HD. Clinician-administered assessments of motor functioning (items from the UHDRS) and standardized objective assessments of cognition (Stroop, Symbol Digit Modalities) were also collected. We examined item bias using differential item functioning (DIF) across HD stage (premanifest, early-, late-) and relative to cognitive performance. We also examined the correlations between self-report and clinician ratings. Regression models that considered total cognitive ability were utilized to determine psychometric reliability of the PROs.

Results

Most PRO items were free from DIF for both staging and cognition. There were modest correlations between PROs and clinician report (ranged from ??0.40 to ??0.60). Modeling analyses indicated that psychometric reliability breaks down with poorer cognition and more progressed disease stage; split-half reliability was compromised (i.e., split-half reliability?<?0.80) when scores were <?136 for Chorea, <?109 for Speech Difficulties, and <?179 for Swallowing Difficulties.

Conclusions

Results indicate that the psychometric reliability of PROs can be compromised as HD symptoms progress and cognition declines. Clinicians should consider PROs in conjunction with other types of assessments when total cognition scores exceed critical thresholds.

     

New measures to capture end of life concerns in Huntington disease: Meaning and Purpose and Concern with Death and Dying from HDQLIFE (a patient-reported outcomes measurement system)

Purpose

Huntington disease (HD) is an incurable terminal disease. Thus, end of life (EOL) concerns are common in these individuals. A quantitative measure of EOL concerns in HD would enable a better understanding of how these concerns impact health-related quality of life. Therefore, we developed new measures of EOL for use in HD.

Methods

An EOL item pool of 45 items was field tested in 507 individuals with prodromal or manifest HD. Exploratory and confirmatory factor analyses (EFA and CFA, respectively) were conducted to establish unidimensional item pools. Item response theory (IRT) and differential item functioning analyses were applied to the identified unidimensional item pools to select the final items.

Results

EFA and CFA supported two separate unidimensional sets of items: Concern with Death and Dying (16 items), and Meaning and Purpose (14 items). IRT and DIF supported the retention of 12 Concern with Death and Dying items and 4 Meaning and Purpose items. IRT data supported the development of both a computer adaptive test (CAT) and a 6-item, static short form for Concern with Death and Dying.

Conclusion

The HDQLIFE Concern with Death and Dying CAT and corresponding 6-item short form, and the 4-item calibrated HDQLIFE Meaning and Purpose scale demonstrate excellent psychometric properties. These new measures have the potential to provide clinically meaningful information about end-of-life preferences and concerns to clinicians and researchers working with individuals with HD. In addition, these measures may also be relevant and useful for other terminal conditions.

     

Effects of perceived autonomy support and basic need satisfaction on quality of life in hemodialysis patients

Purpose

Despite a growing understanding of health-related quality of life (HRQOL) and its determinants in hemodialysis (HD) patients, little is known about the effects and interrelationships concerning the perception of autonomy support and basic need satisfaction of HD patients on their HRQOL. Based on self-determination theory (SDT), this study examines whether HD patients’ perceived autonomy support from health care practitioners (physicians and nurses) relates to the satisfaction of HD patients’ basic needs and in turn influences their HRQOL.

Methods

A questionnaire was administered to 250 Taiwanese HD patients recruited from multiclinical centers and regional hospitals in northern Taiwan. Structural equation modeling (SEM) analysis was conducted to examine the causal relationships between patient perceptions of autonomy support and HRQOL through basic need satisfaction.

Results

The empirical results of SEM indicated that the HD patients’ perceived autonomy support increased the satisfaction of their basic needs (autonomy, competency, and relatedness), as expected. The higher degree of basic need satisfaction led to higher HRQOL, as measured by physical and mental component scores.

Conclusion

Autonomy support from physicians and nurses contributes to improving HD patients’ HRQOL through basic need satisfaction. This indicates that staff caring for patients with severe chronic diseases should offer considerable support for patient autonomy.

     

Testing measurement invariance of the patient-reported outcomes measurement information system pain behaviors score between the US general population sample and a sample of individuals with chronic pain

Purpose

In order to test the difference between group means, the construct measured must have the same meaning for all groups under investigation. This study examined the measurement invariance of responses to the patient-reported outcomes measurement information system (PROMIS) pain behavior (PB) item bank in two samples: the PROMIS calibration sample (Wave 1, N = 426) and a sample recruited from the American Chronic Pain Association (ACPA, N = 750). The ACPA data were collected to increase the number of participants with higher levels of pain.

Methods

Multi-group confirmatory factor analysis (MG-CFA) and two item response theory (IRT)-based differential item functioning (DIF) approaches were employed to evaluate the existence of measurement invariance.

Results

MG-CFA results supported metric invariance of the PROMIS–PB, indicating unstandardized factor loadings with equal across samples. DIF analyses revealed that impact of 6 DIF items was negligible.

Conclusions

Based on the results of both MG-CFA and IRT-based DIF approaches, we recommend retaining the original parameter estimates obtained from the combined samples based on the results of MG-CFA.

     

IRT health outcomes data analysis project: an overview and summary

Background

In June 2004, the National Cancer Institute and the Drug Information Association co-sponsored the conference, “Improving the Measurement of Health Outcomes through the Applications of Item Response Theory (IRT) Modeling: Exploration of Item Banks and Computer-Adaptive Assessment.” A component of the conference was presentation of a psychometric and content analysis of a secondary dataset.

Objectives

A thorough psychometric and content analysis was conducted of two primary domains within a cancer health-related quality of life (HRQOL) dataset.

Research design

HRQOL scales were evaluated using factor analysis for categorical data, IRT modeling, and differential item functioning analyses. In addition, computerized adaptive administration of HRQOL item banks was simulated, and various IRT models were applied and compared.

Subjects

The original data were collected as part of the NCI-funded Quality of Life Evaluation in Oncology (Q-Score) Project. A total of 1,714 patients with cancer or HIV/AIDS were recruited from 5 clinical sites.

Measures

Items from 4 HRQOL instruments were evaluated: Cancer Rehabilitation Evaluation System–Short Form, European Organization for Research and Treatment of Cancer Quality of Life Questionnaire, Functional Assessment of Cancer Therapy and Medical Outcomes Study Short-Form Health Survey.

Results and conclusions

Four lessons learned from the project are discussed: the importance of good developmental item banks, the ambiguity of model fit results, the limits of our knowledge regarding the practical implications of model misfit, and the importance in the measurement of HRQOL of construct definition. With respect to these lessons, areas for future research are suggested. The feasibility of developing item banks for broad definitions of health is discussed.

     

Health-related quality of life of African-American female breast cancer survivors,survivors of other cancers,and those without cancer

Purpose

The purpose of this study was to compare differences in health-related quality of life (HRQOL) between African-American female breast cancer survivors, African-American female survivors of other cancers, and African-American women with no history of cancer.

Methods

Using data from the 2010 National Health Interview Survey (NHIS), the HRQOL of African-American women aged 35 years or older was compared by cancer status. Physical and mental health items from the Patient-Reported Outcomes Measurement Information System (PROMIS) global health scale were used to assess differences in HRQOL.

Results

For summary physical and mental health measures, no significant differences were found between breast cancer survivors and women with no history of cancer; survivors of other cancers reported poorer physical and mental health than did women with no history of cancer. Similar differences were found at the item level. When we examined the two African-American female cancer survivor groups, we found that cancer survivors whose cancer was being treated reported substantially poorer physical health and mental health than did those whose cancer was not being treated. Survivors who had private insurance and were cancer free reported better physical and mental health than did those who did not have private insurance and those who were not cancer free. Breast cancer survivors reported slightly better physical and mental health than did survivors of other cancers.

Conclusions

Our findings highlight the need for public health agencies to adopt practices to improve the mental and physical health of African-American female survivors of cancer.

     

The PROMIS Fatigue<Subscript>FM</Subscript> Profile: a self-report measure of fatigue for use in fibromyalgia

Purpose

Fibromyalgia (FM) is characterized by myriad symptoms and problems. Fatigue is one of the most common, distressing, and disabling symptoms in FM. The purpose of this study was to use fatigue item banks that were developed as part of the Patient-Reported Outcomes Measurement Information System (PROMIS) to devise a self-report measure of fatigue for use in individuals with FM.

Methods

A sample of 532 adults with FM (age range = 18–77, 96.1 % female) completed the PROMIS fatigue item bank. Factor analyses and item response theory analyses were used to identify dimensionality and optimally performing items. These data were used in combination with clinical input to select items for a fatigue self-report measure for use in FM.

Results

Factor analyses revealed four distinct factors in the PROMIS fatigue item bank; items for each univariate subscale were identified by selecting four items with high item information values. A 16-item measure, the PROMIS Fatigue_FM Profile, consisting of four 4-item short forms reflecting fatigue experience (“intensity”) and fatigue impact in three subdomains—social, cognitive, and motivation—was created. The new PROMIS Fatigue_FM Profile short forms showed excellent internal reliability, low ceiling and floor effects, and equivalent or higher test information compared to the standard 4- and 7-item PROMIS fatigue short forms.

Conclusions

The newly developed PROMIS Fatigue_FM Profile, a 16-item measure consisting of four 4-item short forms of self-reported fatigue severity, shows early evidence of good psychometric characteristics, provides the ability to use short forms that assess distinct aspects of fatigue experience and fatigue impact, and demonstrates equivalent or higher levels of test information compared to standard PROMIS fatigue short forms with similar number of items. The PROMIS Fatigue_FM Profile indicated fatigue experience and impact levels approximately 1.5 standard deviations above the normative sample mean across all short forms. Future work to evaluate the validity and reliability of this new measure in individuals with FM is needed.

     

Measurement invariance and general population reference values of the PROMIS Profile 29 in the UK,France, and Germany

Purpose

Comparability of patient-reported outcome measures over different languages is essential to allow cross-national research. We investigate the comparability of the PROMIS Profile 29, a generic health-related quality of life measure, in general population samples in the UK, France, and Germany and present general population reference values.

Methods

A web-based survey was simultaneously conducted in the UK (n?=?1509), France (1501), and Germany (1502). Along with the PROMIS Profile 29, we collected sociodemographic information as well as the EQ-5D. We tested measurement invariance by means of multigroup confirmatory factor analysis (CFA). Differences in the health-related quality of life between countries were modeled by linear regression analysis. We present general population reference data for the included PROMIS domains utilizing plausible value imputation and quantile regression.

Results

Multigroup CFA of the PROMIS Profile 29 showed that factor means are insensitive to potential measurement bias except in one item. We observed significant differences in patient-reported health between countries, which could be partially explained by the differences in overall ratings of health. The physical function and pain interference scales showed considerable floor effects in the normal population in all countries.

Conclusions

Scores derived from the PROMIS Profile 29 are largely comparable across the UK, France, and Germany. Due to the use of plausible value imputation, the presented general population reference values can be compared to data collected with other PROMIS short forms or computer-adaptive tests.

     

Item usage in a multidimensional computerized adaptive test (MCAT) measuring health-related quality of life

Purpose

Examining item usage is an important step in evaluating the performance of a computerized adaptive test (CAT). We study item usage for a newly developed multidimensional CAT which draws items from three PROMIS domains, as well as a disease-specific one.

Methods

The multidimensional item bank used in the current study contained 194 items from four domains: the PROMIS domains fatigue, physical function, and ability to participate in social roles and activities, and a disease-specific domain (the COPD-SIB). The item bank was calibrated using the multidimensional graded response model and data of 795 patients with chronic obstructive pulmonary disease. To evaluate the item usage rates of all individual items in our item bank, CAT simulations were performed on responses generated based on a multivariate uniform distribution. The outcome variables included active bank size and item overuse (usage rate larger than the expected item usage rate).

Results

For average θ-values, the overall active bank size was 9–10%; this number quickly increased as θ-values became more extreme. For values of ?2 and +2, the overall active bank size equaled 39–40%. There was 78% overlap between overused items and active bank size for average θ-values. For more extreme θ-values, the overused items made up a much smaller part of the active bank size: here the overlap was only 35%.

Conclusions

Our results strengthen the claim that relatively short item banks may suffice when using polytomous items (and no content constraints/exposure control mechanisms), especially when using MCAT.

     

Health-related quality of life in haemodialysis patients suffering from chronic itch: results from GEHIS (German Epidemiology Haemodialysis Itch Study)

Purpose

Chronic itch (CI) is a common symptom in haemodialysis (HD) patients affecting about every 4th HD patient. Though it is known that health-related quality of life (HRQOL) is impaired in HD patients, data are lacking on how CI influences HRQOL in this group of patients, also regarding sex differences and the role of sleep and pain.

Methods

In order to investigate the itch prevalence and to investigate associated factors, GEHIS (German Epidemiology Haemodialysis Itch Study) was established as first cross-sectional study being representative for all dialysis units in Germany by considering their regional distribution and size of the located city according to population. HRQOL of 860 HD patients from a randomly selected cluster sample of 25 dialysis units was investigated using patient-reported outcomes as the Short-Form Health Survey (SF-12), the Hospital Anxiety and Depression Scale (HADS) as well as sleep and pain. Additionally, itch-specific QOL was assessed in HD patients affected by CI using the validated ItchyQoL.

Results

42.8 % of HD patients were female. The mean age was 67.2 years (SD ± 13.5). Female patients showed significantly worse values in the mental component summary of the SF-12 as well as in the HADS subscale anxiety compared to male patients. Itch-specific HRQOL did not subject to sex. Patients with impairments in quality of sleep showed significantly worse values in both the SF-12 and the HADS subscales, however not in the ItchyQoL. The occurrence of pain was significantly associated with impaired SF-12, HADS and the ItchyQoL. Multivariate linear regression showed the itch-specific HRQOL to be significantly impaired with rising severity of CI.

Conclusions

Impairments in HRQOL are a serious problem in HD patients. CI means an additional burden in affected patients, also impairing their HRQOL. GEHIS showed significant sex differences in HRQOL outcomes also in HD patients. CI and pain as well as sleep frequently occur combined and significantly impair HRQOL in HD patients. The care of HD patients should be complemented by psychosocial support and mental health services.

     

Reliability and validity of PROMIS measures administered by telephone interview in a longitudinal localized prostate cancer study

Purpose

To evaluate the reliability and validity of six PROMIS measures (anxiety, depression, fatigue, pain interference, physical function, and sleep disturbance) telephone-administered to a diverse, population-based cohort of localized prostate cancer patients.

Methods

Newly diagnosed men were enrolled in the North Carolina Prostate Cancer Comparative Effectiveness and Survivorship Study. PROMIS measures were telephone-administered pre-treatment (baseline), and at 3-months and 12-months post-treatment initiation (N = 778). Reliability was evaluated using Cronbach’s alpha. Dimensionality was examined with bifactor models and explained common variance (ECV). Ordinal logistic regression models were used to detect potential differential item functioning (DIF) for key demographic groups. Convergent and discriminant validity were assessed by correlations with the legacy instruments Memorial Anxiety Scale for Prostate Cancer and SF-12v2. Known-groups validity was examined by age, race/ethnicity, comorbidity, and treatment.

Results

Each PROMIS measure had high Cronbach’s alpha values (0.86–0.96) and was sufficiently unidimensional. Floor effects were observed for anxiety, depression, and pain interference measures; ceiling effects were observed for physical function. No DIF was detected. Convergent validity was established with moderate to strong correlations between PROMIS and legacy measures (0.41–0.77) of similar constructs. Discriminant validity was demonstrated with weak correlations between measures of dissimilar domains (?0.20–?0.31). PROMIS measures detected differences across age, race/ethnicity, and comorbidity groups; no differences were found by treatment.

Conclusions

This study provides support for the reliability and construct validity of six PROMIS measures in prostate cancer, as well as the utility of telephone administration for assessing HRQoL in low literacy and hard-to-reach populations.

     

Linkage between the PROMIS<Superscript>®</Superscript> pediatric and adult emotional distress measures

Purpose

Research studies that measure health-related quality of life (HRQOL) in both children and adults and longitudinal studies that follow children into adulthood need measures that can be compared across these age groups. This study links the PROMIS pediatric and adult emotional distress measures using data from participants with diverse health conditions and disabilities.

Methods

Analyses were conducted and compared in two separate samples to confirm the stability of results. One sample (n = 874) included individuals aged 14–20 years with special health care needs and who require health services. The other sample (n = 641) included individuals aged 14–25 years who have a physical or cognitive disability. Participants completed both PROMIS pediatric and adult measures. Item response theory-based scores were linked using the linear approximation to calibrated projection.

Results

The estimated latent-variable correlation between pediatric and adult PROMIS measures ranged from 0.87 to 0.94. Regression coefficients β ₀ (intercept) and β ₁ (slope), and mean squared error are provided to transform scores from the pediatric to the adult measures, and vice versa.

Conclusions

This study used a relatively new linking method, calibrated projection, to link PROMIS pediatric and adult measure scores, thus expanding the use of PROMIS measures to research that includes both populations.

     

Migrating from a legacy fixed-format measure to CAT administration: calibrating the PHQ-9 to the PROMIS depression measures

Purpose

We provide detailed instructions for analyzing patient-reported outcome (PRO) data collected with an existing (legacy) instrument so that scores can be calibrated to the PRO Measurement Information System (PROMIS) metric. This calibration facilitates migration to computerized adaptive test (CAT) PROMIS data collection, while facilitating research using historical legacy data alongside new PROMIS data.

Methods

A cross-sectional convenience sample (n = 2,178) from the Universities of Washington and Alabama at Birmingham HIV clinics completed the PROMIS short form and Patient Health Questionnaire (PHQ-9) depression symptom measures between August 2008 and December 2009. We calibrated the tests using item response theory. We compared measurement precision of the PHQ-9, the PROMIS short form, and simulated PROMIS CAT.

Results

Dimensionality analyses confirmed the PHQ-9 could be calibrated to the PROMIS metric. We provide code used to score the PHQ-9 on the PROMIS metric. The mean standard errors of measurement were 0.49 for the PHQ-9, 0.35 for the PROMIS short form, and 0.37, 0.28, and 0.27 for 3-, 8-, and 9-item-simulated CATs.

Conclusions

The strategy described here facilitated migration from a fixed-format legacy scale to PROMIS CAT administration and may be useful in other settings.

     

A qualitative study of patients’ perceptions of the utility of patient-reported outcome measures of symptoms in primary care clinics

Purpose

Sleep, pain, anxiety, depression, and low energy/fatigue (SPADE pentad) symptoms are common, but often unrecognized and undertreated in primary care. In an effort to improve symptom outcomes, primary care clinics are integrating patient-reported outcome measures (PROs), such as the Patient-Reported Outcome Measurement Information System (PROMIS), into practice. Whether patients with SPADE symptoms perceive these measures as useful is unknown. The purpose of this qualitative study was to explore patients’ perceptions of the utility of symptom-based PROs in primary care.

Methods

As part of a mixed-methods study, 23 patients (age 24–77 years) with one or more SPADE symptoms participated in one-on-one interviews about the use, implementation and visual display of PROMIS. All interviews were audio-recorded, transcribed, and analyzed using thematic analysis.

Results

Five themes emerged: (1) PROs may foster communication about symptoms; (2) data from PROs may prompt clinical actions; (3) the visual display of symptom scores could be useful for patients and providers, though modifications may enhance interpretation; (4) implementation may vary according to patients’ perception of PRO features (e.g., length); and (5) PROs may be useful, but only to the extent that they are valued, prioritized, and used.

Conclusions

According to patients, the use of PROs to measure symptoms in primary care clinics has the potential to be useful. However, patients are reluctant to complete PROs if they perceive that clinicians do not use the measures to improve care. Barriers to implementation in primary care (e.g., duration of visit) should be addressed to enhance the utility of PROs. Clinical trial registration ClinicalTrials.gov ID: NCT02383862.

     

The challenge of measuring intra-individual change in fatigue during cancer treatment

Purpose

To evaluate how well three different patient-reported outcomes (PROs) measure individual change.

Methods

Two hundred and fourteen patients (from two sites) initiating first or new chemotherapy for any stage of breast or gastrointestinal cancer participated. The 13-item FACIT Fatigue scale, a 7-item PROMIS^® Fatigue Short Form (PROMIS 7a), and the PROMIS^® Fatigue computer adaptive test (CAT) were administered monthly online for 6 months. Reliability of measured change was defined, under a population mixed effects model, as the ratio of estimated systematic variance in rate of change to the estimated total variance of measured individual differences in rate of change. Precision of individual measured change, the standard error of measurement of change, was given by the square root of the rate-of-change sampling variance. Linear and quadratic models were examined up to 3 and up to 6 months.

Results

A linear model for measured change showed the following by 6 and 3 months, respectively: PROMIS CAT (0.363 and 0.342); PROMIS SF (0.408 and 0.533); FACIT (0.459 and 0.473). Quadratic models offered no noteworthy improvement over linear models. Both reliability and precision results demonstrate the need to improve the measurement of intra-individual change.

Conclusions

These results illustrate the challenge of reliably measuring individual change in fatigue with a level of confidence required for intervention. Optimizing clinically useful measurement of intra-individual differences over time continues to pose a challenge for PROs.

     

Evaluating PROMIS<Superscript>®</Superscript> instruments and methods for patient-centered outcomes research: Patient and provider voices in a substance use treatment setting

Purpose

Our work as a primary research site of the Patient-Reported Outcomes Measurement Information System (PROMIS^®), combined with support from the Patient-Centered Outcomes Research Institute, allowed us to evaluate the real-world applicability and acceptability of PROMIS measures in an addiction medicine setting.

Methods

As part of a 3-month prospective observational study, 225 outpatients at a substance abuse treatment clinic completed PROMIS item banks for alcohol use (as well as 15 additional item banks from 8 other PROMIS domains, including emotional distress, sleep, and pain), with assessments at intake, 1-month follow-up, and 3-month follow-up. A subsample of therapists and their patients completed health domain importance ratings and qualitative interviews to elicit feedback regarding the content and format of the patients’ assessment results.

Results

The importance ratings revealed that depression, anxiety, and lack of emotional support were rated highest of the non-alcohol-related domains among both patients and clinicians. General alcohol use was considered most important by both patients and clinicians. Based on their suggestions, changes were made to item response feedback to facilitate comprehension and communication.

Conclusions

Both therapists and patients agreed that their review of the graphical display of scores, as well as individual item responses, helped them to identify areas of greatest concern and was useful for treatment planning. The results of our pilot work demonstrated the value and practicality of incorporating a comprehensive health assessment within a substance abuse treatment setting.

     

Impact of IRT item misfit on score estimates and severity classifications: an examination of PROMIS depression and pain interference item banks

Purpose

In patient-reported outcome research that utilizes item response theory (IRT), using statistical significance tests to detect misfit is usually the focus of IRT model-data fit evaluations. However, such evaluations rarely address the impact/consequence of using misfitting items on the intended clinical applications. This study was designed to evaluate the impact of IRT item misfit on score estimates and severity classifications and to demonstrate a recommended process of model-fit evaluation.

Methods

Using secondary data sources collected from the Patient-Reported Outcome Measurement Information System (PROMIS) wave 1 testing phase, analyses were conducted based on PROMIS depression (28 items; 782 cases) and pain interference (41 items; 845 cases) item banks. The identification of misfitting items was assessed using Orlando and Thissen’s summed-score item-fit statistics and graphical displays. The impact of misfit was evaluated according to the agreement of both IRT-derived T-scores and severity classifications between inclusion and exclusion of misfitting items.

Results

The examination of the presence and impact of misfit suggested that item misfit had a negligible impact on the T-score estimates and severity classifications with the general population sample in the PROMIS depression and pain interference item banks, implying that the impact of item misfit was insignificant.

Conclusions

Findings support the T-score estimates in the two item banks as robust against item misfit at both the group and individual levels and add confidence to the use of T-scores for severity diagnosis in the studied sample. Recommendations on approaches for identifying item misfit (statistical significance) and assessing the misfit impact (practical significance) are given.