Psychometric properties of the communication Confidence Rating Scale for Aphasia (CCRSA): phase 1

Confidence is a construct that has not been explored previously in aphasia research. We developed the Communication Confidence Rating Scale for Aphasia (CCRSA) to assess confidence in communicating in a variety of activities and evaluated its psychometric properties using rating scale (Rasch) analysis. The CCRSA was administered to 21 individuals with aphasia before and after participation in a computer-based language therapy study. Person reliability of the 8-item CCRSA was .77. The 5-category rating scale demonstrated monotonic increases in average measures from low to high ratings. However, one item ("I follow news, sports, stories on TV/movies") misfit the construct defined by the other items (mean square infit = 1.69, item-measure correlation = .41). Deleting this item improved reliability to .79; the 7 remaining items demonstrated excellent fit to the underlying construct, although there was a modest ceiling effect in this sample. Pre- to posttreatment changes on the 7-item CCRSA measure were statistically significant using a paired samples t test. Findings support the reliability and sensitivity of the CCRSA in assessing participants' self-report of communication confidence. Further evaluation of communication confidence is required with larger and more diverse samples.     

A preliminary evaluation of the reliability and validity of a self‐reported communicative functioning item pool

Background: Item banking, an approach to test development based in item response theory (IRT), is beginning to be applied to the measurement of communicative functioning in aphasia. This approach involves calibrating a set of test items responding to a particular latent trait to a common measurement scale. One method for validating such scales is to examine the degree to which obtained item calibration estimates agree with a priori item rankings based on theory or expert opinion.

Aims: The purpose of this study was to evaluate the utility of magnitude estimation (ME) procedures for validating item banks containing self‐reported functional performance items, and to make a preliminary analysis of the validity of a proposed item pool for measuring self‐reported communicative functioning in aphasia.

Methods & Procedures: A total of 14 raters made ME ratings of physical and communicative functioning items. These ratings were evaluated for their intra and inter‐observer reliability and, for subsets of the items, their correspondence with previously published IRT calibration estimates was also evaluated.

Outcomes & Results: Intra‐rater reliability was moderate to high, and inter‐rater reliability was high. Correspondence with IRT calibrations was high for physical items, and moderate for communication items. The distribution of ME ratings for the communication items was negatively skewed.

Conclusions: ME procedures have utility for investigating the validity of functional performance items. The results suggest that communicative functioning may have a more complex latent structure than physical functioning, and that the proposed item pool might benefit from the inclusion of additional items at the lower end of the scale.     

Gavel Club for people with aphasia: communication confidence and quality of communication life

Background: Many factors that contribute to successful living with aphasia intersect with the benefits one can get from attending an aphasia group. Affiliated with Toastmasters International, Gavel Clubs (GCs) for people with aphasia (PWA) provide a range of communication activities that promote public speaking and leadership skills. The constructs of communication confidence and quality of communication life (QCL) were introduced over a decade ago but have not been widely investigated.

Aims: This study aims to investigate the association between weekly participation in GC public speaking activities for PWA and the constructs of QCL and communication confidence. In addition, the study aims to explore the association between the severity of aphasia, QCL and communication confidence.

Methods & Procedures: Eight members of a GC for PWA, who attended 31–33 weekly GC meetings per year between 2012 and 2016, participated in assessments of their QCL (using the ASHA Quality of Communication Life (ASHA QCL)), communication confidence (using the Communication Confidence Rating Scale for Aphasia (CCRSA)), and aphasia severity (using the Western Aphasia Battery-Revised (WAB-R)). A link was sought between severity of aphasia and the constructs of QCL and communication confidence.

Outcomes & Results: QCL improved significantly over four years of participation in the GC (Z = 2.103, n = 8, p = .035, r = .74). Communication confidence also improved significantly (Z = 1.973, p = .049, r = .70). No associations were found between the two scales or between the scales and severity of aphasia.

Conclusions: Weekly participation in GC group activities was associated with improved QCL, as measured by the ASHA QCL, and improved communication confidence as measured by the CCRSA. Decision-making (measured by the CCRSA) and the Roles and Responsibilities domain (in the ASHA QCL) also improved. More research is needed to verify these findings using a study design that includes a control group, to identify the links between the various psychosocial aspects at play in the life of PWA and to conceptualise how the gains achieved with GC participation can be applied more broadly to successful living with aphasia.     

Evidence-based measurement: which disability scale for neurologic rehabilitation?

OBJECTIVE: To compare the 10-item Barthel Index (BI), 18-item Functional Independence Measure (FIM), and 30-item Functional Independence Measure + Functional Assessment Measure (FIM+FAM) as measures of disability outcomes for neurologic rehabilitation. METHODS: A total of 149 inpatients from two rehabilitation units in South England specializing in neurologic disorders were studied. Traditional psychometric methods were used to evaluate and compare acceptability (score distributions), reliability (internal consistency, intrarater reproducibility), validity (concurrent, convergent and discriminant construct), and responsiveness (standardized response mean). RESULTS: All three rating scales satisfied recommended criteria for reliable and valid measurement of disability, and are acceptable and responsive in this study sample. The FIM and FIM+FAM total scales are psychometrically similar measures of global disability. The BI, FIM, and FIM+FAM motor scales are psychometrically similar measures of physical disability. The FIM and FIM+FAM cognitive scales are psychometrically similar measures of physical disability. CONCLUSIONS: In the sample studied, the BI, FIM, FIM+FAM have similar measurement properties, when examined using traditional psychometric analyses. Although instruments with more items and item response categories generate more qualitative information about an outcome, they may not improve its measurement. Results highlight the importance of using recognized techniques of scale construction to develop health outcome measures.     

Exploring the factor structure of the La Trobe Communication Questionnaire: Insights into the nature of communication deficits following traumatic brain injury

Background: Self and close other reports of communication ability can provide a time‐efficient means of evaluating conversational discourse after traumatic brain injury (TBI). The La Trobe Communication Questionnaire (LCQ) measures perceived communication ability from various sources including self‐perceptions and perceptions of others. Content and test–retest reliability and discriminant validity of the LCQ have been demonstrated previously with adults following TBI.

Aims: This study was undertaken to explore the factor structure of the LCQ as revealed within the data collected from 88 adults with severe TBI and their close others.

Methods & Procedures: Construct validity was examined using a Principal Component Factor Analytic procedure with Varimax rotation.

Outcomes & Results: A seven‐factor structure that accounted for 60.88% of the variance was revealed. Of the LCQ items, 27 clearly loaded on to one of the seven communication factors that were identified.

Conclusions: The factor structure that emerged depicts the multidimensional nature of conversation and the complex interplay between cognitive and communication processes that social discourse demands. These results support the construct validity of the LCQ and indicate that it can be used as a reliable and valid measure of communication ability after severe TBI.     

Validation of the Revised Behavior Summarized Evaluation Scale

The Behavioral Summarized Evaluation scale (BSE), previously published and validated, was developed for the evaluation of the autistic behavior in developmentally disordered children. A revised version of this scale, the Revised Behavior Summarized Evaluation Scale (BSE-R) completed the 20-item BSE scale with the most relevant items extracted from a similar evaluation carried out with very young children. Thus 9 items were added to the original scale concerning nonverbal communication, emotional, and perception areas. This paper reports the reliability and validity studies of this new scale. In addition to confirming the previously published findings concerning the first version of the BSE, new items were extracted from the BSE-R content validity study. They involve fundamental functions such as intention and imitation which open new perspectives for a physiopathological approach to developmental disorders. The BSE-R is a useful tool for progressive recording of the evolution of patients both treated over long periods and included in short-term controlled therapeutic studies.     

Exploring the use of the Dementia Management Strategies Scale in caregivers of persons with dementia in Singapore

Background: Well-being in persons with dementia (PWD) depends much on the quality and type of care received. The Dementia Management Strategies Scale (DMSS) is a useful instrument to appraise care styles of caregivers. The present study expanded on previous research by refining and establishing the scale's content validity and psychometric properties in the Singapore context. Method: Five family caregivers and four dementia care professionals (nurse, occupational therapist, social worker and doctor) reviewed the DMSS for content validity. Two hundred and forty-six family caregivers completed questionnaires which assessed caregiver and patient characteristics, and dementia management strategies with DMSS. Internal consistency reliability was assessed and construct validity was evaluated through Pearson's correlation with extant instruments. Results: Eight items from the 28-item DMSS were omitted after content review as they were deemed inappropriate in our socio-cultural setting. A factor analysis with Varimax rotation confirmed a two-factor structure (positive and negative dimensions) for the revised DMSS (rDMSS). The two subscales showed good internal consistency (Cronbach's alpha .89 and .87). Moderate to strong correlations (.35–.53) with the scales, Zarit Burden Instrument, Revised Memory and Behavioural Problems Checklist, General Health Questionnaire, Short Sense of Competence Scale, Gains in Alzheimer's Care Instrument and Positive Aspects of Caregiving established convergent and divergent construct validity of rDMSS. Conclusion: The shortened 20-item rDMSS is a psychometrically valid instrument which can serve as a measure of dementia care strategy from the perspective of the caregiver in Singapore.     

Assessment of patient interpersonal behavior: Development and validation of a rating scale

Objective: Patient in-session interpersonal behavior, as part of the therapeutic alliance, is an important aspect of the psychotherapy process and impacts treatment outcome. In the present study, the development and validation of a rating scale of patient in-session interpersonal behavior is described. Method: A 10-item rating scale, the Assessment Form of Patient Interpersonal Behavior (AFPIB), was developed using an inductive procedure. The AFPIB was then validated in a sample of patients with hypochondriasis (N = 30), by having two independent raters assess patients’ interpersonal behaviors shown in videotaped psychotherapy sessions (N = 60). Results: The AFPIB demonstrated good reliability and validity. Conclusions: Thus, the AFPIB seems to be a promising rating scale for the assessment of patient interpersonal behavior shown in psychotherapy sessions.     

Measuring communicative functioning in community‐dwelling stroke survivors: Conceptual foundation and item development

Background: Studies employing item response theory methods to evaluate communicative functioning assessment items have found that a broad range of communication tasks and activities may fit a unidimensional measurement model, but that additional item content is needed to extend the range of ability effectively measured by the small subset of items that have been evaluated.

Aims: To describe the item identification, evaluation, and development process used to substantiate the content relevance and representativeness of a set of communicative functioning assessment items targeting community‐dwelling stroke survivors.

Methods & Procedures: Electronic and secondary references were searched to identify assessment tools with item content designed to measure communicative functioning in adults with neurogenic communication disorders. Candidate items were evaluated using face‐to‐face interviewer‐assisted survey groups conducted independently with communicatively impaired stroke survivors (n = 59) and their communicative partners (n = 61). Web‐based surveys were employed to evaluate candidate items from the perspective of practising speech‐language pathologists (n = 114).

Outcomes & Results: A total of 673 items were identified from 33 instruments. A total of 426 met the specified concept definition; 211 were determined to be non‐redundant; 166 were identified by key stakeholders as unambiguous, relevant, and moderately to very important to daily functioning.

Conclusions: The item pool developed samples a representative range of communication behaviours, activities, and life situations that are relevant to community‐dwelling stroke survivors. Further research using item response theory methods is required to substantiate the construct dimensionality and range of ability effectively measured by the item pool, and to evaluate dynamic assessment algorithms designed to minimise response burden.     

Computer-based treatment of poststroke language disorders: a non-inferiority study of telerehabilitation compared to in-person service delivery

Background: Telerehabilitation promises to greatly expand access of underserved populations to speech therapy, but concerns remain about the effectiveness of services delivered remotely compared to in-person treatment.

Aims: To evaluate the effectiveness of telerehabilitation, we conducted a randomized non-inferiority trial for chronic poststroke communication disorders, testing whether equivalent gains can be expected from in-person vs. telerehabilitative clinical service delivery, with both groups completing homework exercises outside of therapist contact time.

Methods & Procedures: We treated 44 participants with aphasia or cognitive-linguistic communication disorder (CLCD). Treatment comprised tablet-based homework exercises and realistic, customized treatment plans tailored to the needs of each individual client. Clients had weekly 1-h sessions with the therapist over 10 weeks, with the interaction randomized to in-person and telerehabilitation conditions. Objective gains were assessed with the Western Aphasia Battery aphasia quotient (WAB-AQ) (for aphasia) and Cognitive-Linguistic Quick Test (CLQT) (for CLCD) and subjective gains with the Communication Confidence Rating Scale for Aphasia (CCRSA) (self-rating) and Communication Effectiveness Index (CETI) (partner rating).

Outcomes & Results: Participants improved significantly on all of these measures, with statistically equivalent gains between in-person and telerehabilitation groups for WAB-AQ, CLQT, and CETI. Only the CCRSA showed an advantage for the in-person group. Gains on WAB-AQ were correlated with total time spent on offline exercises.

Conclusions: Clinician-guided computer-based treatment is effective for producing widespread gains in language and communication skills in chronic stroke. Linguistic gains are equivalent whether clinician services are provided via telerehabilitation equipment or in person. Communicative confidence may still benefit from in-person treatment, reinforcing the need for social engagement in addition to deficit-focused linguistic treatment.     

A scale of attitudes toward the application of eugenics to the treatment of people with mental retardation

As part of a larger study of the attitudes of diverse samples towards the application of eugenics to the treatment of people with mental retardation, a 32-item summated rating scale was developed as a contemporary, brief, easy to administer and score, and psychometrically sound instrument. Data were collected and analysed that indicated satisfactory item characteristics and reliability, and initial support for the content and construct validities of the scale. Analyses of social desirability data revealed that scale scores were not influenced by the subjects' desire to adhere to socially desirable expectations. The scale should be useful for the investigation of questions concerning the formation, structure and correlates of attitudes toward the application of eugenics to the treatment of people with mental retardation, and the relationship of these attitudes to contemporary mental retardation policies and practices.     

Reliability of a method for establishing the capacity of individuals with an intellectual disability to respond to Likert scales

Abstract

Background?The study reported here was an examination of the reliability of a method for determining acquiescent responding and the capacity to respond to items using a Likert scale response format by adults with an intellectual disability.

Method?Reliability of the outcomes of these procedures was investigated using a test–retest design. Associations with receptive vocabulary were examined.

Results?The majority of the participants did not demonstrate acquiescent responding. Individuals’ responses to the Likert-type discrimination tasks were consistent, although this varied somewhat depending upon the abstractness of the task. There was some association between receptive language age equivalence scores and respondent performance.

Conclusion?It is recommended that the pretest protocol (a) be modified to improve its reliability, and (b) this modified version be used with study participants who have an intellectual disability to ascertain the appropriate level of choice to be used for items that use a Likert response format.     

Validity of the Malaise Inventory in general population samples

Background: The Malaise Inventory is a commonly used self-completion scale for assessing psychiatric morbidity. There is some evidence that it may represent two separate psychological and somatic sub-scales rather than a single underlying factor of distress. This paper provides further information on the factor structure of the Inventory and on the reliability and validity of the total scale and two sub-scales. Methods: Two general population samples completed the full Inventory: over 11,000 subjects from the National Child Development Study at ages 23 and 33, and 544 mothers of adolescents included in the Isle of Wight epidemiological surveys. Results: The internal consistency of the full 24-item scale and the 15-item psychological sub-scale were found to be acceptable, but the eight-item somatic sub-scale was less reliable. Factor analysis of all 24 items identified a first main general factor and a second more purely psychological factor. Receiver operating characteristic (ROC) analysis indicated that the validity of the scale held for men and women separately and for different socio-economic groups, by reference to external criteria covering current or recent psychiatric morbidity and service use, and that the psychological sub-scale had no greater validity than the full scale. Conclusions: This study did not support the separate scoring of a somatic sub-scale of the Malaise Inventory. Use of the 15-item psychological sub-scale can be justified on the grounds of reduced time and cost for completion, with little loss of reliability or validity, but this approach would not significantly enhance the properties of the Inventory by comparison with the full 24-item scale. Inclusion of somatic items may be more problematic when the full scale is used to compare particular sub-populations with different propensities for physical morbidity, such as different age groups, and in these circumstances it would be a sensible precaution to utilise the 15-item psychological sub-scale. Accepted: 9 March 1999     

The Emotional Processing Scale: Scale refinement and abridgement (EPS-25)

Objective

The Emotional Processing Scale (EPS) is a 38-item, eight-factor self-report questionnaire designed to measure emotional processing styles and deficits. Scale development is an ongoing process and our aim was to (i) refine the scale by trying out items from a new item pool and (ii) shorten the scale to enhance its clinical and research utility.

Methods

Fifteen new items were added to the original 38-item pool. The resulting 53-item scale was administered to four groups (N=690) (mental health, healthy controls, pain patients, and general medical practice attendees). Exploratory factor analysis was used to explore the underlying factor structure.

Results

Maximum likelihood (ML) factor analysis was used to guide the process of item selection and scale reduction. Four of the previous eight factors remained in similar form, two of the original factors were discarded, and one new factor emerged incorporating items from two previous factors. The revised version of the scale (EPS-25) has a 25-item five-factor structure. Internal reliability was moderate to high for all five factors.

Conclusion

The psychometric properties of the revised scale appear promising, particularly in relation to the detection of differences between diagnostic groups.     

The Development,Reliability, and Validity of the Social Impact of Repetitive Behavior Scale in Children with Autism Spectrum Disorder

ABSTRACT

Introduction: Current measures of restrictive and repetitive behavior (RRB) in people with autism focus on severity and intensity and, to some degree, the global interference of the behavior. In this study we developed the Social Impact of Repetitive Behavior Scale (SIRBS) to capture several different contexts in which repetitive behavior is likely to occur and interfere.

Methods: SIRBS items were selected through reviewing the RRB literature, participant chart reviews, and consensus among authors, followed by an initial piloting and further refinement of the tool. Caregivers completed the SIRBS a total of 400 times.

Results: Subscales showed high internal consistency and good test-retest reliability, moderate concurrent validity, and average to excellent inter-rater reliability.

Conclusion: The SIRBS is a psychometrically reliable and valid measure of the social impact of repetitive behavior with children with autism. Additional research is needed to independently validate it and conduct an initial exploratory factor analysis of subscales.     

The MBT Adherence and Competence Scale (MBT-ACS): Development,structure and reliability

Abstract

The properties of the 17-item Mentalization-Based Treatment Adherence and Competence Scale (MBT-ACS) were investigated in a reliability study in which 18 psychotherapy sessions, comprising two sessions by nine different therapists, were rated by seven different raters. The overall reliabilities for adherence and competence for seven raters were high, .84 and .88 respectively. The level of reliability declined by number of raters but was still acceptable for two raters (.60 and .68). The reliabilities for the various items differed. The MBT-ACS was found to be an appropriate rating measure for treatment fidelity and useful for the purposes of quality control and supervision. The reliability may be enhanced by redefining some items and reducing their numbers.