Psychometric evaluation of the Finnish translation of the assessment of communication and interaction skills (ACIS-FI)

Abstract

Background: The Model of Human Occupation (MOHO) is a widely used conceptual practice model in Finland. Therefore, Finnish translations of valid and reliable MOHO assessments are needed.

Objective: The primary objective of this study was to examine the psychometric properties of the Finnish translation of the Assessment of Communication and Interaction Skills (ACIS-FI) using the many-facet Rasch model approach.

Methods and Materials: Twenty-eight occupational therapists completed an in-person training workshop on the ACIS-FI and participated as raters in this study. One hundred and forty-eight clients were rated using the ACIS-FI. Rating scale functioning, unidimensionality, person validity and rater severity, item targeting, and item and person separation statistics were examined.

Results: The rating scales demonstrated adequate functioning; the rating category ‘deficit’ was infrequently adopted by the raters. The ACIS-FI had satisfactory construct validity, as confirmed by all items exhibiting unidimensionality within a single construct (i.e. communication and interaction skills), and appropriate item fit. Validity was further confirmed through low person misfit (6%) and low rater misfit (3.6%). No ceiling or floor effects were found. The ACIS-FI was able to separate clients into four levels of communication and interaction skills.

Conclusions: This study offers evidence for the validity of the ACIS-FI as a measure of communication and interaction skills in occupational therapy.

Significance: The ACIS-FI offers Finnish practitioners and researchers a valid tool to measure communication and interactions skills that is theoretically grounded in the MOHO.     

Psychometric evaluation of an assessment of awareness using two different Rasch models

Abstract

Objective: The aim of this study was to evaluate the psychometric properties of the Assessment of Awareness of Disability (AAD), applied to a sample of clients with a variety of diagnoses, using two different Rasch models. Methods: The sample consisted of 245 clients admitted to occupational therapy services in Sweden. The psychometric properties of the AAD measures were analysed and compared using both Rasch partial credit (PCM) and rating scale (RSM) models. Results: In relation to rating scale properties and person response validity, the RSM yielded better results. However, the PCM demonstrated better item goodness-of-fit statistics than the RSM, supporting the internal scale validity of the AAD. Some systematic indications of sample-dependent misfit were also found in the data. Conclusion: The results support the validity of the AAD in a sample of clients with various diagnoses, although some limitations of the tool are addressed. The outcomes also demonstrate the importance of making cautious choices when selecting a Rasch model for investigating psychometric properties, as the choice has a substantial impact on the interpretation of the findings.     

A Rasch Analysis Study Determining the Internal Validity and Separation Reliability of the Inventory of Reading Occupations - Adult: A Measure of Functional Literacy Participation

Abstract

The objective of this study was to determine the internal validity and separation reliability of the Inventory of Reading Occupations – Adult (IRO-Adult), an assessment of functional literacy participation for adults. Participants included 69 adults with typical reading abilities and difficulties with reading. The fit of the IRO-Adult items was analyzed with the Rasch model of unidimensionality using goodness-of-fit analysis and analysis of standardized residuals. Also analyzed was the tool’s ability to reliably separate distinct levels of reading participation and the optimal functioning of its rating scales. The IRO-Adult items fit the Rasch model of unidimensionality. Three test items can be further refined to strengthen internal validity. Items are able to reliably detect multiple levels of reading participation and change in functional literacy participation over time. Several categories in the tool’s rating scale can be combined for better clarity and to optimize rating scale functioning. The study provides preliminary evidence of validity and reliability of the IRO-Adult as a useful tool to assess functional literacy participation of adults. Further studies on other psychometric properties and clinical utility are warranted.     

Rasch Analysis of the Michigan Hand Questionnaire

ObjectivesThe Michigan Hand Questionnaire (MHQ) is a commonly used evaluation for hand problems, but previous work reports conflicting evidence regarding the subscale structures. Rasch analysis uses probabilistic modeling of items and responses: if scale items can be fit to the Rasch model, it provides evidence of construct validity and interval-level measurement for precise statistical estimates. We conducted Rasch analysis on the MHQ to evaluate model fit, unidimensionality of the subscales, bias across person factors, and conversion to interval metrics.MethodsWe conducted a secondary Rasch analysis of MHQ data from 924 persons with thumb basal joint osteoarthritis using the RUMM2030 software. Modeling was based on responses for the most affected hand and person factors including age, sex, type of work, whether the dominant side was the most affected, and surgical status. The analysis plan followed the published recommendations for examinations of person and item fit, with iterative adjustments as required.ResultsA total of 11 of the 37 items required rescoring to create orderly progression of scoring thresholds. Only the overall hand function and pain subscales could be fit to the Rasch model, demonstrating unidimensionality and good reliability of fit estimates. Dividing the activities of daily living subscale into unilateral and bilateral activities also allowed unilateral activities to fit the model. Persistent misfitting in other subscales suggested local dependency and response bias across multiple person factors.ConclusionsThis Rasch analysis of the MHQ raises concerns regarding the validity and fundamental measurement properties of this widely used outcome evaluation when used as a summary score.     

Rasch Analysis of the General Self-Efficacy Scale in Workers with Traumatic Limb Injuries

Purpose The purpose of this study was to apply Rasch analysis to examine the unidimensionality and reliability of the General Self-Efficacy Scale (GSE) in workers with traumatic limb injuries. Furthermore, if the items of the GSE fitted the Rasch model’s assumptions, we transformed the raw sum ordinal scores of the GSE into Rasch interval scores. Methods A total of 1076 participants completed the GSE at 1 month post injury. Rasch analysis was used to examine the unidimensionality and person reliability of the GSE. The unidimensionality of the GSE was verified by determining whether the items fit the Rasch model’s assumptions: (1) item fit indices: infit and outfit mean square (MNSQ) ranged from 0.6 to 1.4; and (2) the eigenvalue of the first factor extracted from principal component analysis (PCA) for residuals was <2. Person reliability was calculated. Results The unidimensionality of the 10-item GSE was supported in terms of good item fit statistics (infit and outfit MNSQ ranging from 0.92 to 1.32) and acceptable eigenvalues (1.6) of the first factor of the PCA, with person reliability = 0.89. Consequently, the raw sum scores of the GSE were transformed into Rasch scores. Conclusions The results indicated that the items of GSE are unidimensional and have acceptable person reliability in workers with traumatic limb injuries. Additionally, the raw sum scores of the GSE can be transformed into Rasch interval scores for prospective users to quantify workers’ levels of self-efficacy and to conduct further statistical analyses.     

Validation of the Italian version of the Client Satisfaction with Device module of the Orthotics and Prosthetics Users' Survey

BackgroundInformation on patient satisfaction with orthosis (PSwO) is crucial for verifying and enhancing orthotic quality, for clinical decision making, and for improving patient's quality of life.ObjectiveTo perform the translation and cross-cultural adaptation into Italian of the recently revised version of the Client Satisfaction with Device (CSD) module of the Orthotics and Prosthetics Users' Survey, and then analyze its psychometric properties using factor and Rasch analyses.MethodsWe translated and cross-culturally adapted the revised CSD into Italian (CDS-It) and assessed it in a convenience sample of orthotic-user patients with orthopedic, neurological and rheumatic conditions (N = 178; 56% men; median age, 62 years). Exploratory factor analysis and Rasch analysis (rating scale model) were used to investigate, respectively, dimensionality and metric properties of the scale.ResultsFactor analysis confirmed the substantial unidimensionality of the CSD-It. The rating scale fulfilled the category functioning criteria. All items fitted the Rasch model except #2 (“The weight of my device is manageable”) that overfitted the model, and #4 (“It is easy to put on my device”) that was underfitting in six stroke patients (i.e. not systematically). The targeting of item difficulty to person ability was out of range. The person separation reliability was 0.70 and Cronbach's alpha 0.73. The residual correlation between items #7 and #8 showed a borderline local dependency.ConclusionsThis study confirms the validity of the CSD-It, and provides a useful starting point for further refinement of this outcome measure.     

Application of Rasch analysis in health care is increasing and is applied for variable reasons in mobility instruments

ObjectiveTo identify the frequency of Rasch analysis use in health instrument development or refinement and the characteristics of Rasch application in mobility scales.Study Design and SettingThe entire databases of Medline, CINAHL, PEDro, EMBASE, Cochrane Central Register of Controlled Trials, and Cochrane Database of Systematic Reviews were searched until January 2009. Articles that reported the development or refinement of health instruments using Rasch analysis were included. Of the 234 articles that met inclusion, 10 were categorized as “mobility” instruments. Data were extracted relating to each instrument and the use of Rasch analysis in the development or refinement of the instruments.ResultsThe number of articles reporting the use of Rasch analysis of health instruments is increasing, from 1 article in 1987 to 48 articles in 2007. Of the 10 mobility instruments examined, the primary reason Rasch was used varied. Reasons included assessing instrument unidimensionality, differential item functioning, rating categories, item hierarchy, and redundant items.ConclusionThe application of Rasch analysis in health instrument development has markedly increased in recent years. However, few mobility instruments have been developed or refined using Rasch analysis. The reasons that the Rasch model was used varied across mobility instruments.     

Psychometric properties of the Mastery Scale-Chinese version: Applying classical test theory and Rasch analysis

Abstract

Aims: This study examines the reliability and validity of the Mastery Scale-Chinese version (MS-C) when applied to three groups diagnosed with major depression, schizophrenia, or HIV/AIDS. Methods: The individuals participating in the study were recruited from outpatient units of a medical center and a municipal hospital in northern Taiwan. The study sample (n = 2009) included 237 patients with depressive disorders, 160 with schizophrenia, and 1612 with HIV/AIDS. The reliability and construct validity of the MS-C was evaluated by confirmatory factor analysis (CFA) and Rasch analysis. Results: The results of the CFA showed that the MS-C has adequate construct validity with all indices meeting the criteria, except for chi-square values. The Rasch analysis supported the four-point rating scale structure and a unidimensional construct of the MS-C. The DIF analysis showed that all items demonstrated stable measurement properties among the different diagnosis groups (major depression, schizophrenia, HIV/AIDS). Conclusion: This study found that MS-C has acceptable psychometric qualities in terms of reliability, construct validity, rating scale performance, and item characteristics when applied to patients with depression, schizophrenia, and HIV/AIDS in Taiwan.     

Evaluation of the Social Participation Questionnaire in adult patients with depressive symptoms using Rasch analysis

Purpose

The aim of this study was to explore the psychometric properties of the 22-item Social Participation Questionnaire (SPQ).

Methods

The SPQ was administered to 789 adult primary care patients with depressive symptoms. As the items were intended to be summed together to provide total score, Rasch analysis (partial credit model) was applied to assess the overall fit of the model, individual item fit, differential item functioning (DIF), targeting of persons, response dependency, unidimensionality and person separation.

Results

To improve the scale’s fit, it was necessary to re-score the response format. Two items demonstrated some DIF for gender and eight items showed DIF for age. To support the assumption of unidimensionality post hoc principal component analysis was performed. The analysis showed two subtests of the residuals with positive and negative loadings, but the person estimates derived from these two subtests were not statistically different to that derived from all items taken together. The response dependence between two items was identified; however, the magnitude of difficulty was very small. Although the questionnaire appeared to have insufficient items to assess the full spectrum of informal social contact, the SPQ was reasonably well targeted.

Conclusion

The SPQ is a promising questionnaire for the measurement of social participation although it could benefit from the inclusion of further items to measure informal social contact. This study found support for the internal validity, internal consistency reliability, and unidimensionality. A future study will investigate whether targeting can be improved when additional items are included.     

The measurement properties of the Occupational Circumstances Interview and Rating Scale - Sweden (OCAIRS-S V2)

Abstract

Aim: This paper reports the measurement properties of the Occupational Circumstances Assessment Interview and Rating Scale - Sweden (OCAIRS-S V2). The OCAIRS is a semi-structured interview and rating scale designed to capture, in detail, a person's occupational participation. The English version 4.0 has been translated into Swedish. The psychometric properties of the Swedish version are unknown. Methods: Eleven Swedish occupational therapists working in mental health completed 38 OCAIRS-S (V2) assessments in addition to linking videotapes. A total of 60 clients were, therefore, entered into the analysis. Many-faceted Rasch analysis was used to analyse the data. Results and conclusions: Results supported internal, construct, and person response validity of the OCAIRS-S (V2). Inter-rater reliability was established. The scale was shown to discriminate between people who were living in the community. In addition, the results indicate a need to explore the skill items.     

Living with Chronic Illness Scale: International validation through the classic test theory and Rasch analysis among Spanish‐speaking populations with long‐term conditions

BackgroundThe Living with Chronic Illness (LW‐CI) Scale is a comprehensive patient‐reported outcome measure that evaluates the complex process of living with long‐term conditions.ObjectiveThis study aimed to analyse the psychometric properties of the LW‐CI scale according to the classic test theory and the Rasch model among individuals living with different long‐term conditions.DesignThis was an observational, international and cross‐sectional study.MethodsA total of 2753 people from six Spanish‐speaking countries living with type 2 diabetes mellitus, chronic obstructive pulmonary disease, chronic heart failure, Parkinson''s disease, hypertension and osteoarthritis were included. The acceptability, internal consistency and validity of the LW‐CI scale were analysed using the classical test theory, and fit to the model, unidimensionality, person separation index, item local independency and differential item functioning were analysed using the Rasch model.ResultsCronbach''s α for the LW‐CI scale was .91, and correlation values for all domains of the LW‐CI scale ranged from .62 to .68, except for Domain 1, which showed correlation coefficients less than .30. The LW‐CI domains showed a good fit to the Rasch model, with unidimensionality, item local independency and moderate reliability providing scores in a true interval scale. Except for two items, the LW‐CI scale was free from bias by long‐term condition type.DiscussionAfter some adjustments, the LW‐CI scale is a reliable and valid measure showing a good fit to the Rasch model and is ready for use in research and clinical practice. Future implementation studies are suggested.Patient and Public ContributionPatient and public involvement was conducted before this validation study ‐ in the pilot study phase.     

An evaluation of the structural validity of the shoulder pain and disability index (SPADI) using the Rasch model

Purpose

The shoulder pain and disability index (SPADI) has been extensively evaluated for its psychometric properties using classical test theory (CTT). The purpose of this study was to evaluate its structural validity using Rasch model analysis.

Methods

Responses to the SPADI from 1030 patients referred for physiotherapy with shoulder pain and enrolled in a prospective cohort study were available for Rasch model analysis. Overall fit, individual person and item fit, response format, dependence, unidimensionality, targeting, reliability and differential item functioning (DIF) were examined.

Results

The SPADI pain subscale initially demonstrated a misfit due to DIF by age and gender. After iterative analysis it showed good fit to the Rasch model with acceptable targeting and unidimensionality (overall fit Chi-square statistic 57.2, p?=?0.1; mean item fit residual 0.19 (1.5) and mean person fit residual 0.44 (1.1); person separation index (PSI) of 0.83. The disability subscale however shows significant misfit due to uniform DIF even after iterative analyses were used to explore different solutions to the sources of misfit (overall fit (Chi-square statistic 57.2, p?=?0.1); mean item fit residual 0.54 (1.26) and mean person fit residual 0.38 (1.0); PSI 0.84).

Conclusions

Rasch Model analysis of the SPADI has identified some strengths and limitations not previously observed using CTT methods. The SPADI should be treated as two separate subscales. The SPADI is a widely used outcome measure in clinical practice and research; however, the scores derived from it must be interpreted with caution. The pain subscale fits the Rasch model expectations well. The disability subscale does not fit the Rasch model and its current format does not meet the criteria for true interval-level measurement required for use as a primary endpoint in clinical trials. Clinicians should therefore exercise caution when interpreting score changes on the disability subscale and attempt to compare their scores to age- and sex-stratified data.

     

Examining the validity and reliability of the revised developmental work personality scale (RDWPS)-traditional Chinese version

BackgroundWork personality was found to relate to successful work adjustment, job readiness, work motivation and job maintenance. The revised developmental work personality scale (RDWPS) is a self-reported assessment to evaluate the work personality of the examinee which further psychometric study is required and needs to be applied to different culture. The aim of this study was to examine the theoretical structure, validity, and reliability of the traditional Chinese version of the RDWPS.MethodsThe subjects were 113 university students with no known physical or mental illness who aged between 19 and 22. Cluster analysis was used to examine the theoretical structure. Rasch analysis was applied to examine the psychometric properties.ResultsWe dropped the unfit three items and found the 11 items of traditional Chinese version of the RDWPS was adequate to fit the theoretical construct. The results of the Rasch analysis showed that the 11 items with a three-point rating scale of the traditional Chinese version of the RDWPS had acceptable internal consistency (α = 0.76), moderate reliability, and met the criteria of unidimensionality.ConclusionsThe results of this study provided initial evidence that the traditional Chinese version of the RDWPS can be readily applied to research related to work behavior for those whose primary language is traditional Chinese, with adequate reliability and validity.     

The ADL taxonomy for persons with mental disorders – adaptation and evaluation

Background: There is a lack of occupation-focused instruments to assess Activities of Daily Living (ADL) that are intended for persons with mental disorders. The ADL Taxonomy is an instrument that is widely-used within clinical practice for persons with physical impairment. The aim of this study was to adapt the ADL Taxonomy for persons with mental disorders and evaluate its validity.

Methods: An expert group of Occupational Therapists (OTs) from psychiatric care adapted the ADL Taxonomy to fit the client group, including creating three new items. OTs in psychiatric care collected client data and evaluated the instrument for usability. Rasch analysis was used to evaluate the contruct validity of 16 activities separately.

Results: The OTs collected 123 assessments from clients with various mental disorders. Ten activities had excellent, and four had acceptable, psychometric properties with regard to item and person fit and unidimensionality. The activity managing the day/time gave complex results and would benefit from further development. The OTs found the test version intelligible, relevant and easy to use.

Conclusions: The ADL Taxonomy for persons with mental disorders has 16 activities with three to six actions each, and is now ready for clinical use.     

Measuring Sexual Risk for HIV: A Rasch Scaling Approach

In this study, we developed an HIV transmission risk scale and examined its psychometric properties using data on sexual behavior obtained from a probability sample of adult men who have sex with men living in Chicago. We used Messick’s (Am Psychol 50:741–749, 1995) conceptualization of unified validity theory to organize the psychometric properties of data. Evidence related to scale content was investigated via Rasch item fit statistics, point-measure correlations, and expert evaluation. The substantive aspect of validity was addressed by interpreting the meaningfulness of the item difficulty hierarchy (continuum of risky behaviors) and assessment of person fit. The structural aspect of validity was assessed using Rasch item fit statistics, principal component analysis of standardized residuals, and other residual analyses. The generalizability aspect of validity was investigated via internal consistency reliability estimates for both items and persons, and aspects of external validity were addressed by examining between-group differences with respect to levels of risky behavior. Applications and suggested future studies are discussed.     

The calibrated,unidimensional anxiety item bank for cardiovascular patients provided the basis for anxiety assessment in cardiovascular rehabilitation patients

ObjectiveComputer adaptive tests (CATs) offer a flexible, test fair, and economic opportunity for accurate measurement of anxiety in patients with cardiovascular diseases (CVDs). The objective of this study was to develop and calibrate an item bank [anxiety item bank for cardiovascular patients (AIB-cardio)] as a prerequisite for an anxiety-CAT in CVD patients.Study Design and SettingAfter pretesting for relevance and comprehension, a pool of 155 anxiety items was answered on a five-point Likert scale. Sample consisted of 715 CVD patients, who were recruited in 14 German cardiac rehabilitation centers. A confirmatory factor analysis (CFA), Mokken analysis, and Rasch analysis were conducted.ResultsThe results of CFA and Mokken analysis confirmed one factor structure and double monotonicity. In Rasch analysis, merging response categories and removing items with misfit, differential item functioning or local response dependency reduced the AIB-cardio to 37 items. The AIB-cardio fitted to the Rasch model with a nonsignificant item–trait interaction (chi-square, 133.89; degrees of freedom, 111; P = 0.07). Person separation reliability was 0.85, and unidimensionality could be verified.ConclusionThe calibrated, unidimensional AIB-cardio provides the basis for a CAT to assess anxiety in rehabilitation patients with CVD with good psychometric properties. Further testing in other cardiovascular patients is needed to increase generalizability.     

Rasch analysis of the Personal Wellbeing Index

Purpose

The Personal Wellbeing Index (PWI) is used internationally to measure subjective well-being. While numerous studies have reported on the psychometric properties of the PWI using classic test theory, the current paper provides additional psychometric evaluation using the Rasch measurement model.

Method

The responses to the PWI items of 593 healthy adults living in Australia or Canada were analysed using Rasch analysis. Assessed were overall model fit was assessed, individual person fit and item fit, reliability, differential item functioning (DIF), unidimensionality, targeting, and response format.

Results

Results indicate excellent psychometric properties of the PWI, provided the religion/spirituality item is excluded. The seven-item PWI showed good model fit (overall item–trait interaction χ ² = 70.59, df = 63, p = 0.24), excellent person separation (PSI = .89), no item or person misfit, and no DIF for country or gender, and unidimensionality was supported. In addition, only very mild disordered thresholds were observed for the ‘safety’ item, indicating that overall, the 11-point response format was suitable for this population.

Conclusion

It is recommended that the religion/spirituality item not be included in the aggregated or averaged total score in Western context and that caution should be used in comparison across different groups where the eight-item version has been used.

     

An examination of the psychometric properties of the occupational identity questionnaire for community-living elderly who require care

IntroductionAuthors created an Occupational Identity Questionnaire Provisional version (OIQ-P) to assess occupational identity for elderly individuals. The purpose of this study was to examine the psychometric properties of the OIQ-P.MethodsParticipants included 135 (42 males) elderly who lived locally and required care or support. OIQ-P was evaluated in terms of structural validity, criterion validity and internal consistency.ResultsBased on the results of an exploratory factor analysis and confirmatory factor analysis, an OIQ with a factor structure of 3 factors and 14 items was created. Rasch rating scale model revealed that 14 participants and 1 item did not fit the goodness of fit, nevertheless, the overall result was good. Spearman''s rank correlation coefficient indicates that there was a law correlation between OIQ and the occupational identity scale of the Occupational Performance History Interview Version 2. In terms of internal consistency, the person separation index and person separation reliability coefficient were 2.30 and 0.84, respectively.ConclusionThis study confirmed the structural validity, criterion validity and internal consistency for the OIQ. To enhance the clinical utility of the OIQ, it is necessary to examine the interpretability and conduct an intervention study using the OIQ.     

Everyday technology use among older adults in Sweden and Portugal

Abstract

Background: Technology use is relevant for engaging in everyday occupations and aging in place and difficulties might limit participation.

Aim: The aim of the present study was two-fold: to investigate the internal scale validity of the Everyday Technology Use Questionnaire (ETUQ) in a matched Portuguese and a Swedish sample of older adults without known cognitive impairments and to compare everyday technology use (i.e., the relevance of Everyday Technology (ET) and perceived difficulty of using ET).

Methods: The Rasch rating scale model was used to investigate the psychometric properties of the data from the two countries. Analyses were then used to compare the samples.

Results: There is evidence of internal scale and person response validity in the Swedish and Portuguese ETUQ datasets. The Swedish sample demonstrated almost 25% more ETs to be relevant, as compared to the Portuguese sample (p?<?.05). The samples’ means of perceived difficulty in using ETs were similar.

Conclusions: The results support that the ETUQ is a valid assessment for measuring the relevance of and perceived difficulty of ET use and is able to detect differences and similarities between older adults in these European countries in relation to ET use in the home and society.     

Construct validity of a revised version of the Occupational Balance Questionnaire

Abstract

Background: Occupational balance is fundamental to occupational therapy and occupational science. Therefore, the Occupational Balance Questionnaire (OBQ) was developed and has previously been found psychometrically valid according to classical test theory.

Aim: To investigate the internal construct validity of the OBQ using Rasch measurement theory.

Material and methods: Data from two general population samples were used to investigate the psychometric properties of the OBQ according to Rasch measurement theory.

Results: The analyses identified problems with the current response scale and multidimensionality of two items. As a result, a revised version, the OBQ11, was suggested and exhibited response categories that worked properly, good reliability (0.92), model fit and measurement invariance across age and gender groups. The hierarchical item ordering was in agreement with previous research.

Conclusion: The new OBQ11 satisfies the measurement criteria defined by the Rasch model. However, further studies of additional samples are needed to validate its generic properties.

Significance: The purpose of the OBQ11 is to measure occupational balance of individuals or groups, and to identify aspects of occupational balance in need of improvement. Our observations suggest that the OBQ11 is a valid and promising complement to other instruments.