Using item response theory in the development and validation of the College-Oriented Eating Disorders Screen

The current study examined the psychometric characteristics of the College-Oriented Eating Disorders Screen (COEDS), a college-student-focused screening measure to assess and identify individuals at-risk for the development of eating disordered pathology. By screening a large pool of pilot questions and using methods based in item response theory (IRT), seven items were identified with well-targeted contents that discriminated well across the continuum of eating disorder severity. The resulting measure evidenced a unidimensional factor structure and correlated highly with the original COEDS, standard measures of eating disorders pathology, and a measure of associated symptomatology (e.g., depressive symptoms). Based on these results, we discuss the utility of the COEDS as a prognostic indicator for risk of eating disordered pathology among college students.     

基于项目反应理论的中国职业人群职业紧张测量核心量表条目分析

目的 利用项目反应理论（item response theory,IRT）对《中国版职业紧张核心量表》质量进行分析与评价,为后期量表使用和修订提供参考依据。方法 采用方便抽样方法,抽取湖北省两家三甲医院和多家一、二级医院共1261名医务人员作为研究对象,应用《中国版职业紧张核心量表》调查其职业紧张情况。采用主成分分析验证量表4个维度的单维性。采用IRT中的Same Jima等级反应模型计算每个条目的区分度、难度系数和信息量,从微观角度评价量表的测量特性。结果 量表4个维度均满足单维性假设。IRT结果显示所有条目的区分度较好,取值范围在0.67~3.10;17个条目中有13个条目的难度系数在-2.78~2.30之间,且不存在难度逆反现象,条目9和11难度过高且难度逆反,条目15和16难度过低过高并存且有难度逆反现象,提示待改进;除了条目9、11和15提供的信息量中等,条目16和17提供的信息量较差以外,其余条目的信息量均较好。结论 《中国版职业紧张核心量表》所有条目的区分度较好。从难度系数和信息量两个角度,条目9、11、15、16、17的测验质量均是有待改进的,其余条目性能良好,建议针对上述分析结果结合专家意见对问题条目进行修订。     

Health status assessment for the twenty-first century: item response theory, item banking and computer adaptive testing

Health status assessment is frequently used to evaluate the combined impact of human immunodeficiency virus (HIV) disease and its treatment on functioning and well-being from the patient's perspective. No single health status measure can efficiently cover the range of problems in functioning and well-being experienced across HIV disease stages. Item response theory (IRT), item banking and computer adaptive testing (CAT) provide a solution to measuring health-related quality of life (HRQoL) across different stages of HIV disease. IRT allows us to examine the response characteristics of individual items and the relationship between responses to individual items and the responses to each other item in a domain. With information on the response characteristics of a large number of items covering a HRQoL domain (e.g. physical function, and psychological well-being), and information on the interrelationships between all pairs of these items and the total scale, we can construct more efficient scales. Item banks consist of large sets of questions representing various levels of a HRQoL domain that can be used to develop brief, efficient scales for measuring the domain. CAT is the application of IRT and item banks to the tailored assessment of HRQoL domains specific to individual patients. Given the results of IRT analyses and computer-assisted test administration, more efficient and brief scales can be used to measure multiple domains of HRQoL for clinical trials and longitudinal observational studies.     

基于经典测量理论与项目反应理论的慢性胃炎生命质量测定量表QLICD-CG(V2.0)条目分析

目的 应用经典测量理论与项目反应理论对慢性胃炎患者生命质量量表QLICD-CG(V2.0)的条目进行分析。方法 采用QLICD-CG(V2.0)量表,对163名慢性胃炎患者进行生命质量评估。利用Multilog 7.03软件进行项目反应理论分析得出每个条目的难度、区分度系数和信息量,同时结合经典测量理论分析的4种统计方法来评价条目质量的优劣。结果 CTT结果显示,除了3个条目（GPH3、GPS3、CG11）外,剩余条目都符合4种统计学方法至少满足3种的标准;IRT结果显示,所有条目的难度系数都在-6.42～4.36,而且随着难度等级（B1→B4）增加呈现出单调递增的趋势,所有条目的区分度都在1.37～1.69,所有条目的平均信息量都在0.356～0.780。39个条目中,37个条目的性能良好,2个条目（GPH3、GPS3）需要优化。结论 QLICD-CG(V2.0)量表的大部分条目的性能较好,但少数条目仍需进一步改进。     

The feasibility of applying item response theory to measures of migraine impact: A re-analysis of three clinical studies

Background: Item response theory (IRT) is a powerful framework for analyzing multiitem scales and is central to the implementation of computerized adaptive testing. Objectives: To explain the use of IRT to examine measurement properties and to apply IRT to a questionnaire for measuring migraine impact – the Migraine Specific Questionnaire (MSQ). Methods: Data from three clinical studies that employed the MSQ-version 1 were analyzed by confirmatory factor analysis for categorical data and by IRT modeling. Results: Confirmatory factor analyses showed very high correlations between the factors hypothesized by the original test constructions. Further, high item loadings on one common factor suggest that migraine impact may be adequately assessed by only one score. IRT analyses of the MSQ were feasible and provided several suggestions as to how to improve the items and in particular the response choices. Out of 15 items, 13 showed adequate fit to the IRT model. In general, IRT scores were strongly associated with the scores proposed by the original test developers and with the total item sum score. Analysis of response consistency showed that more than 90% of the patients answered consistently according to a unidimensional IRT model. For the remaining patients, scores on the dimension of emotional function were less strongly related to the overall IRT scores that mainly reflected role limitations. Such response patterns can be detected easily using response consistency indices. Analysis of test precision across score levels revealed that the MSQ was most precise at one standard deviation worse than the mean impact level for migraine patients that are not in treatment. Thus, gains in test precision can be achieved by developing items aimed at less severe levels of migraine impact. Conclusions: IRT proved useful for analyzing the MSQ. The approach warrants further testing in a more comprehensive item pool for headache impact that would enable computerized adaptive testing.     

基于经典测量理论与项目反应理论的宫颈癌生命质量测定量表QLICP-CE（V2.0）条目分析

目的 应用 CTT 与 IRT 两种分析理论对宫颈癌患者生命质量量表（QLICP-CE V2.0）的条目进行分析与评价。 方法 通过应用 QLICP-CE（V2.0）对 186 例宫颈癌病人进行测评,采用经典测量理论 CTT 中的四种统计方法（变异度法、相关系数法、因子分析法、克朗巴赫系数法）来评价条目质量的好坏。同时采用项目反应理论IRT中的 Samejima 等级反应模型计算每个条目的难度、区分度系数和信息量。 结果 CTT 分析结果提示 QLICP-CE（V2.0）共性模块中有 9 个条目与其所在领域的相关性比较低,而特异模块中有3个。IRT结果显示所有条目的区分度较好,取值范围均在0.64～1.33;44个条目中有35个条目的难度系数取值范围在-3.49～3.76,且随着难度等级（B1→B4）的增加呈现出单调递增的趋势;除 3 个条目外所有条目的平均信息量均较好。 结论 QLICP-CE（V2.0）量表所有条目区分度比较好,大部分条目的性能良好,但仍然有少部分条目有待进一步修订并验证效果。     

Calibration of an item pool for assessing the burden of headaches: An application of item response theory to the Headache Impact Test (HIT™)

Background: Measurement of headache impact is important in clinical trials, case detection, and the clinical monitoring of patients. Computerized adaptive testing (CAT) of headache impact has potential advantages over traditional fixed-length tests in terms of precision, relevance, real-time quality control and flexibility. Objective: To develop an item pool that can be used for a computerized adaptive test of headache impact. Methods: We analyzed responses to four well-known tests of headache impact from a population-based sample of recent headache sufferers (n = 1016). We used confirmatory factor analysis for categorical data and analyses based on item response theory (IRT). Results: In factor analyses, we found very high correlations between the factors hypothesized by the original test constructers, both within and between the original questionnaires. These results suggest that a single score of headache impact is sufficient. We established a pool of 47 items which fitted the generalized partial credit IRT model. By simulating a computerized adaptive health test we showed that an adaptive test of only five items had a very high concordance with the score based on all items and that different worst-case item selection scenarios did not lead to bias. Conclusion: We have established a headache impact item pool that can be used in CAT of headache impact.     

The minimal clinically important difference determined using item response theory models: an attempt to solve the issue of the association with baseline score

ObjectivesDetermining the minimal clinically important difference (MCID) of questionnaires on an interval scale, the trait level (TL) scale, using item response theory (IRT) models could overcome its association with baseline severity. The aim of this study was to compare the sensitivity (Se), specificity (Sp), and predictive values (PVs) of the MCID determined on the score scale (MCID-Sc) or the TL scale (MCID-TL).Study Design and SettingThe MCID-Sc and MCID-TL of the MOS-SF36 general health subscale were determined for deterioration and improvement on a cohort of 1,170 patients using an anchor-based method and a partial credit model. The Se, Sp, and PV were calculated using the global rating of change (the anchor) as the gold standard test.ResultsThe MCID-Sc magnitude was smaller for improvement (1.58 points) than for deterioration (−7.91 points). The Se, Sp, and PV were similar for MCID-Sc and MCID-TL in both cases. However, if the MCID was defined on the score scale as a function of a range of baseline scores, its Se, Sp, and PV were consistently higher.ConclusionThis study reinforces the recommendations concerning the use of an MCID-Sc defined as a function of a range of baseline scores.     

Item banking to improve, shorten and computerize self-reported fatigue: An illustration of steps to create a core item bank from the FACIT-Fatigue Scale

Fatigue is a common symptom among cancer patients and the general population. Due to its subjective nature, fatigue has been difficult to effectively and efficiently assess. Modern computerized adaptive testing (CAT) can enable precise assessment of fatigue using a small number of items from a fatigue item bank. CAT enables brief assessment by selecting questions from an item bank that provide the maximum amount of information given a person's previous responses. This article illustrates steps to prepare such an item bank, using 13 items from the Functional Assessment of Chronic Illness Therapy Fatigue Subscale (FACIT-F) as the basis. Samples included 1022 cancer patients and 1010 people from the general population. An Item Response Theory (IRT)-based rating scale model, a polytomous extension of the Rasch dichotomous model was utilized. Nine items demonstrating acceptable psychometric properties were selected and positioned on the fatigue continuum. The fatigue levels measured by these nine items along with their response categories covered 66.8% of the general population and 82.6% of the cancer patients. Although the operational CAT algorithms to handle polytomously scored items are still in progress, we illustrated how CAT may work by using nine core items to measure level of fatigue. Using this illustration, a fatigue measure comparable to its full-length 13-item scale administration was obtained using four items. The resulting item bank can serve as a core to which will be added a psychometrically sound and operational item bank covering the entire fatigue continuum.     

The PROMIS Physical Function item bank was calibrated to a standardized metric and shown to improve measurement efficiency

ObjectiveTo document the development and psychometric evaluation of the Patient-Reported Outcomes Measurement Information System (PROMIS) Physical Function (PF) item bank and static instruments.Study Design and SettingThe items were evaluated using qualitative and quantitative methods. A total of 16,065 adults answered item subsets (n > 2,200/item) on the Internet, with oversampling of the chronically ill. Classical test and item response theory methods were used to evaluate 149 PROMIS PF items plus 10 Short Form-36 and 20 Health Assessment Questionnaire-Disability Index items. A graded response model was used to estimate item parameters, which were normed to a mean of 50 (standard deviation [SD] = 10) in a US general population sample.ResultsThe final bank consists of 124 PROMIS items covering upper, central, and lower extremity functions and instrumental activities of daily living. In simulations, a 10-item computerized adaptive test (CAT) eliminated floor and decreased ceiling effects, achieving higher measurement precision than any comparable length static tool across four SDs of the measurement range. Improved psychometric properties were transferred to the CAT's superior ability to identify differences between age and disease groups.ConclusionThe item bank provides a common metric and can improve the measurement of PF by facilitating the standardization of patient-reported outcome measures and implementation of CATs for more efficient PF assessments over a larger range.     

慢性病生命质量量表共性模块项目反应理论分析

目的运用项目反应理论(IRT)对慢性病患者生命质量测定量表共性模块(QLICD-GM)条目进行分析,筛选信息量较高条目。方法应用QLICD-GM测评7种慢性病患者620例,采用塞姆吉玛等级反应模型计算每个条目的难度、区分度系数和信息量,绘制项目特征曲线;根据平均信息量筛选条目;采用MULTILOG 7.0软件进行计算和作图。结果QLICD-GM共性模块29个条目的区分度均为1.2～1.9;难度(程度)均呈严格单调递增,取值范围为-3.05～2.18;依据平均信息量,结合条目特征筛选保留24个条目。结论QLICD-GM各条目区分度均较好、选项设置合理、难度合适,分析模型选择正确;项目反应理论可筛选出信息量较高条目,弥补经典测量理论(CTT)的不足。     

Computerized adaptive test for patients with foot or ankle impairments produced valid and responsive measures of function

Objective  We tested the item response theory (IRT) model assumptions of the original item bank, and evaluated the practical and psychometric adequacy, of a computerized adaptive test (CAT) for patients with foot or ankle impairments seeking rehabilitation in outpatient therapy clinics. Methods  Data from 10,287 patients with foot or ankle impairments receiving outpatient physical therapy were analyzed. We first examined the unidimensionality, fit, and invariance IRT assumptions of the CAT item bank. Then we evaluated the efficiency of the CAT administration and construct validity and sensitivity of change of the foot/ankle CAT measure of lower-extremity functional status (FS). Results  Results supported unidimensionality, model fit, and invariance of item parameters and patient ability estimates. On average, the CAT used seven items to produce precise estimates of FS that adequately covered the content range with negligible floor and ceiling effects. Patients who were older, had more chronic symptoms, had more surgeries, had more comorbidities, and did not exercise prior to receiving rehabilitation reported worse discharge FS. Seventy-one percent of patients obtained statistically significant change at follow-up. Change of 8 FS units (scale 0–100) represented minimal clinically important improvement. Conclusions  We concluded that the foot/ankle item bank met IRT assumptions and that the CAT FS measure was precise, valid, and responsive, supporting its use in routine clinical application.     

Having a fit: impact of number of items and distribution of data on traditional criteria for assessing IRT’s unidimensionality assumption

Purpose  Confirmatory factor analysis fit criteria typically are used to evaluate the unidimensionality of item banks. This study explored the degree to which the values of these statistics are affected by two characteristics of item banks developed to measure health outcomes: large numbers of items and nonnormal data. Methods  Analyses were conducted on simulated and observed data. Observed data were responses to the Patient-Reported Outcome Measurement Information System (PROMIS) Pain Impact Item Bank. Simulated data fit the graded response model and conformed to a normal distribution or mirrored the distribution of the observed data. Confirmatory factor analyses (CFA), parallel analysis, and bifactor analysis were conducted. Results  CFA fit values were found to be sensitive to data distribution and number of items. In some instances impact of distribution and item number was quite large. Conclusions  We concluded that using traditional cutoffs and standards for CFA fit statistics is not recommended for establishing unidimensionality of item banks. An investigative approach is favored over reliance on published criteria. We found bifactor analysis to be appealing in this regard because it allows evaluation of the relative impact of secondary dimensions. In addition to these methodological conclusions, we judged the items of the PROMIS Pain Impact bank to be sufficiently unidimensional for item response theory (IRT) modeling.     

Items and dimensions for the construction of a multidimensional computerized adaptive test to measure fatigue in patients with rheumatoid arthritis

ObjectivesDevelopment of an item pool to construct a future computerized adaptive test (CAT) for fatigue in rheumatoid arthritis (RA). The item pool was based on the patients' perspective and examined for face and content validity previously. This study assessed the fit of the items with seven predefined dimensions and examined the item pool's dimensionality structure in statistical terms.Study Design and SettingA total of 551 patients with RA participated in this study. Several steps were conducted to come from an explorative item pool to a psychometrically sound item bank. The item response theory (IRT) analysis using the generalized partial credit model was conducted for each of the seven predefined dimensions. Poorly fitting items were removed. Finally, the best possible multidimensional IRT (MIRT) model for the data was identified.ResultsIn IRT analysis, 49 items showed insufficient item characteristics. Items with a discriminative ability below 0.60 and/or model misfit effect sizes greater than 0.10 were removed. Factor analysis on the 196 remaining items revealed three dimensions, namely severity, impact, and variability of fatigue. The dimensions were further confirmed in MIRT model analysis.ConclusionThis study provided an initially calibrated item bank and showed which dimensions and items can be used for the development of a multidimensional CAT for fatigue in RA.     

Group leader and participant perceptions of Balancing Everyday Life,a group-based lifestyle intervention for mental health service users

Abstract

Background: There exist few recovery and occupation-based interventions for mental health service users. Balancing Everyday Life (BEL) is a new occupation-based lifestyle intervention that was created to fill this need.

Aim: To gain group leaders’ and participants’ perspectives of the BEL intervention content and format, including factors that helped, hindered, and could be improved.

Methods: A constructivist grounded theory method guided data collection and analysis. Interviews took place with 12 BEL group leaders and 19 BEL participants from out-patient psychiatry settings and community-based day centers in Sweden.

Results: BEL’s structure and content were appreciated, yet flexibility was desired to adapt to participant needs. BEL could act as a bridge, helping participants connect with others, and to a more engaged and balanced everyday life. Facilitating factors included a person-focused (versus illness-focused) approach, physical and emotional environments, and connection. Barriers included room resources. More sessions were desired for the intervention.

Conclusion: Group leaders and participants experienced BEL as a useful tool to instigate meaningful change and connection in the participants’ lives. The combination of a positive person-focused approach and group support was appreciated. These results could inform future research, evaluation, and development of occupation-focused lifestyle interventions for mental health service users.     

A comparison of traditional and Rasch cut points for assessing clinically important change in health-related quality of life among patients with asthma

Background Patient-perceived change in health-related quality of life (HRQoL) domains has often been classified using a 15-point patient transition rating scale. However, traditional change levels of trivial ( − 1, 0, or 1), minimal (2, 3 or − 2, − 3), moderate (4, 5 or − 4, − 5) and large (6, 7 or − 6, − 7) on this scale have been arbitrarily defined and originally assumed that change related to an improvement was the same as that for a decline. Objective To compare traditional and Rasch partial credit model-derived cut points and the mean changes for each change categorization when assessing clinically important change in asthma-specific HRQoL. Methods Our sample included 396 asthmatic outpatients who completed bimonthly telephone interviews on the Asthma Quality of Life Questionnaire and transition rating items over 1 year of participation. We employed item response theory in a novel approach to identify cut points on domain-specific HRQoL change data and transition ratings. After determining natural cut points for minimal, moderate, and large differences on the transition rating anchor, we calculated mean changes under change categorizations for both improvements and declines for the two transition rating classification approaches. Results Although traditional and Rasch categorizations for small, moderate, and large changes slightly differed and displayed a lack of symmetry between improvements and declines, nearly all mean changes between classification approaches were comparable. Conclusions In this study, traditional transition rating cut points remain suitable to assess HRQoL clinical significance in outpatients with asthma. An earlier version of this paper was presented at the International Society for Quality of Life Research Symposium, June 29, 2004, in Boston, MA and at the International Society for Quality of Life Annual Meeting, October 20, 2005, in San Francisco, CA.     

Additional psychometric information and vision-specific questionnaires are available for age-related macular degeneration

Purpose  To present psychometric information and studies dealing with questionnaires for age-related macular degeneration (AMD) and visually impaired patients in addition to the study by Finger et al. “Quality of life in AMD: a review of available vision-specific psychometric tools”. We propose that their literature search should not have focused solely on the specific eye disease AMD. Methods  The literature search was partly replicated (PubMed) by using “visual impairment” instead of “macular degeneration” as free text words. Psychometric information was obtained from the additional studies. Preliminary results from a differential item functioning (DIF) analysis used to examine the relationship between item responses on the Vision-related quality of life Core Measure (VCM1) of AMD patients versus patients with other eye conditions are discussed. Results  Eight studies of visually impaired patient populations, including AMD patients, are discussed, with psychometric information from six vision-specific questionnaires. The VCM1 items did not present DIF, which means that the items were equally interpreted by all patients. Conclusions  The results on DIF and the additional studies presented here confirm that a specific eye disorder is of minor importance in the choice of a vision-specific questionnaire or, in this case, a literature search.     

Association between vaccination status,symptom identification and healthcare use: Implications for test negative design observational studies

AimTo test the internal validity of the test-negative design (TND) by investigating associations between maternal influenza vaccination, and new virus detection episodes (VDEs), acute respiratory illness, and healthcare visits in their children.MethodsEighty-five children from a birth cohort provided daily symptoms, weekly nasal swabs, and healthcare use data until age 2-years. Effect estimates are summarised as incidence rate ratios (IRR).ResultsThere was no association between maternal vaccination and VDEs in children (IRR = 1.1; 95 %CI = 0.9–1.2). Influenza-vaccinated mothers were more likely than unvaccinated mothers to both report, and seek healthcare for, acute lower respiratory illness in their children, IRR = 2.4; 95 %CI = 1.2–4.8 and IRR = 2.2; 95 %CI = 1.1–4.3, respectively.ConclusionA key assumption of the TND, that healthcare seeking behaviour for conditions of the same severity is not associated with vaccine receipt, did not hold. Further studies of the performance of the TND in different populations are required to confirm its validity.     

Mode effects in the center for epidemiologic studies depression (CES-D) scale: personal digital assistant vs. paper and pencil administration

As interest grows in creating computerized versions of established paper-and-pencil (P&P) questionnaires, it becomes increasingly important to explore whether changing the administration modes of questionnaires affects participants' responses. This study investigated whether mode effects exist when administering the Center for Epidemiologic Studies Depression (CES-D) scale by a personal digital assistant (PDA) versus the classic P&P mode. The Differential Functioning of Items and Tests (DFIT) procedure identified mode effects on the overall test and individual items. A mixed-effects regression model summarized the mode effects in terms of CES-D scores, and identified interactions with covariates. When the P&P questionnaire was administered first, scores were higher on average (2.4-2.8 points) than those of the other administrations (PDA second, PDA first, and P&P second), and all 20 questionnaire items exhibited a statistically significant mode effect. Highly educated people and younger people demonstrated a smaller difference in scores between the two modes. The mode-by-order effect influenced the interpretation of CES-D scores, especially when screening for depression using the established cut-off scores. These results underscore the importance of evaluating the cross-mode equivalence of psychosocial instruments before administering them in non-established modes.