Reliability of a Rating Scale to Assess Post-stroke Psychiatric Symptoms

Background : Various rating scales for post-stroke psychiatric symptoms such as cognitive impairment have long been used in drug efficacy trials in Japan. However, their reliability has not been established. The purpose of the present study was to examine the inter-rater and intra-rater reliability of a rating scale, which has been conventionally used in Japan.
Methods : The most frequent symptoms, including apathy, emotional and cognitive impairment, found in post-stroke patients were assessed using a rating scale comprising 17 items and 4 global assessment scales. Semi-structured interviews of 18 patients with symptoms with various degrees of severity were videotaped. Twelve physicians who were not interviewers independently assessed the severity of symptoms in the 18 patients by replaying the videotapes. This procedure was repeated twice with an interval of two months to examine inter-rater and test-retest reliability.
Results : The results revealed almost satisfactory reliability. Intraclass correlation coefficients were over 0.6 in the inter-rater analysis for most items and the concordance correlation coefficients were over 0.7 in the intra-rater analysis. The rating scale was considered to be reliable, although some items showed relatively low agreement.
Conclusion : The conventional rating scale to assess post-stroke psychiatric symptoms showed satisfactory inter-rater and intra-rater reliability by the videotape method. The validity study should be further investigated.     

Assessing impairment in patients with panic disorder: the Sheehan Disability Scale

Summary The DSM-III-R incorporates both distress (symptoms) and disability (impairment) in the definition of a psychiatric disorder. In psychiatric research there is a wide array of instruments used to measure symptom severity, but a limited selection for the assessment of impairment. The psychometric properties of one such instrument, The Sheehan Disability Scale (Sheehan 1983), are evaluated in this paper. The data analyzed come from two studies of patients with panic disorder, the Cross National Collaborative Panic Study-Phase I and the Panic Depression Study. In this report both the alpha coefficients and factor analyses indicate that the reliability of the scale is acceptable. The factor structure of the items and the sensitivity to change of their composite demonstrate satisfactory construct validity. The criterion-related validity is substantiated by the significant relationship between symptomatology and impairment. These analyses were limited to patients with panic disorder. Further work is needed to evaluate the instrument in assessing patients with other disorders.     

Improving assessment in small fiber neuropathy

Interval measures at the impairment level addressing symptoms and at the activity/participation level addressing daily and social restrictions have not been developed for small fiber neuropathy (SFN). We developed an SFN‐specific Rasch‐built overall disability scale (SFN‐RODS©), an activity/participation scale at the interval level. A preliminary SFN‐RODS containing 146 activity/participation items was assessed twice (reliability studies) in 238 patients with SFN. The ordinal‐based 13‐item SFN‐symptoms inventory questionnaire (SFN‐SIQ©) and pain‐visual‐analogue‐scale were also assessed (validity studies). The pre‐SFN‐RODS and SFN‐SIQ data were subjected to the Rasch analyses. The pre‐SFN‐RODS did not meet Rasch model expectations. Based on requirements, such as misfit statistics, differential item functioning, and local dependency, items were systematically removed and model fit improved. Finally, a 32‐item SFN‐RODS© scale was constructed that fulfilled all Rasch requirements, demonstrating acceptable reliability and validity scores. The 13‐item SFN‐SIQ© was successfully transformed to an interval Rasch‐built measure fulfilling model's requirements. In conclusion, the 32‐item SFN‐RODS© is a disease‐specific interval measure suitable for detecting activity limitations and participation restrictions in patients with SFN. The 13‐item SFN‐SIQ© was transformed through Rasch to an interval measure. The use of these scales is recommended in future clinical interventional trials involving patients with SFN.     

Validity of the 36-item Persian (Farsi) version of the world health organization disability assessment schedule (WHODAS) 2.0

The purpose of this study was to validate a self-administered 36-item Persian (Farsi) version of the World Health Organization (WHO) Disability Assessment Schedule II (now referred to as WHODAS 2.0) for assessment of psychiatric patients’ perceptions of their functioning and disability. WHODAS 2.0 items were analyzed using two approaches. Reliability, consistency, and factor structure were assessed using Cronbach’s α and factor analysis, and item response theory (IRT) was used to determine how well the WHODAS 2.0 items fitted the Rasch paradigm. Data were collected from 614 psychiatric outpatients in Tehran. The mean overall disability score for the sample was 37.57. The scale had excellent reliability (Cronbach’s α?=?0.94). The IRT-based analysis showed that overall the set of items had a poor fit to the Rasch paradigm; the exceptions were items belonging to domains D1 (cognition), D2 (mobility), and D5 (life activities). There were several problematic items associated with dimensions D3 (self-care) and D4 (getting along). There were at least two badly fitted items associated with all dimensions. This study is the first to examine the psychometric properties of the self-administered, 36-item, Persian version of WHODAS 2.0 in psychiatric outpatients. This version has acceptable reliability and validity in psychiatric patients, but a reformulation of problematic items and further validation tests would be required to produce a robust measurement instrument.     

A Delusion Assessment Scale for Psychotic Major Depression: Reliability, Validity, and Utility

BACKGROUND: Although delusions are the hallmark of major depression with psychotic features, a scale to measure the intensity of beliefs across multiple delusional domains in this condition has been unavailable. The development and assessment of the Delusional Assessment Scale (DAS) are described. METHODS: Scale items were selected initially based on previous studies of delusional ideation in schizophrenia. A three-point item to assess mood congruence was added. A 15-item scale was assessed in 92 subjects participating in the four-site collaborative study of the pharmacotherapy of major depression with psychotic features. Maximum likelihood method was used to determine scale factors. The internal consistency of these factors was determined. Comparisons between scale scores and ratings from the Brief Psychiatric Rating Scale (BPRS) (Overall and Gorham 1962) were used to assess convergent and discriminant validity. RESULTS: The data were fit by a five-factors model (impact, conviction, disorganization, bizarreness, and extension). Inter-rater reliability of the five factors ranged from .77 for conviction and .74 for impact to .37 for disorganization. Internal consistency for each of the five factors was > or =.72. Scores on specific domains were significantly correlated with the BPRS unusual thought content item and positive symptom subscale scores. CONCLUSIONS: The DAS is a reliable measure of 5 delusional domains.     

Clinical gait and balance scale (GABS): validation and utilization

Gait and Balance Scale (GABS) consists of historical information and examination of 14 different gait and balance parameters designed to assess the severity of these functional domains. Thirty-five patients with Parkinson's disease (PD), Hoehn and Yahr stages 1-3, were tested during their "off" period. GABS items were compared to quantitative data from two computerized gait analysis instruments, GAITRite and Pro Balance Master. Intra-class correlation coefficients were calculated to establish reliability. Intra-rater test-retest reliability was determined using Cohen's Kappa statistic. Concurrent validity was derived using the Spearman's rho test with the items from GABS, GAITRite and Balance Master. Intra-rater reliability was high with k>0.41 (k=kappa statistic) for 17 items, 6 had k>0.61. When performing validity measurements, a number of items on the GABS had a correlation coefficient significant at p<0.01 (2-tailed). Posture, pull test, balance during stance, single limb stance, tandem stance, turning, toe walking and functional reach had significant correlation with Balance Master data (R=0.46-1). Gait, arm swing, gait speed, steps/5 m, 'up-and-go test', modified performance oriented assessment of gait scale and provocative testing had significant correlation with the GAITRite items (R=0.51-0.83). GABS is an easy-to-use comprehensive clinical scale with high intra-rater and internal item reliability. We have shown concurrent validity with two computerized gait analysis instruments. We expect GABS to have a particular utility in clinical trials designed to modify functional impairment associated with abnormalities in gait and balance.     

Psychometric evaluation and tests of validity of the Medical Outcomes Study 12-item Sleep Scale (MOS sleep)

ObjectiveTo validate the psychometric properties of the Medical Outcomes Study (MOS) Sleep Scale in subjects with restless legs syndrome (RLS).MethodsData from a clinical trial program involving two Phase III, double-blind, placebo-controlled trials of ropinirole in subjects with moderate-to-severe primary RLS were analyzed. Subjects were assessed on the MOS Sleep Scale at baseline, Weeks 8 and 12.ResultsThe baseline validation population included 551 subjects on which full longitudinal data are available. Psychometric assessment of four MOS sleep domains revealed satisfactory item convergent validity (r > 0.40) for most items. All domain items in both trials surpassed the standard for item discriminant validity, with no significant floor or ceiling effects. The MOS sleep domain scores showed good internal consistency reliability. Concurrent validity (r = 0.40) was exceeded in correlations between the RLS overall quality-of-life score and sleep problems index II. The clinical validity of the MOS Sleep Scale was demonstrated against self-reported RLS symptoms and clinician-determined severity; changes in MOS Sleep Scale were responsive to improvements in RLS severity, as measured by the Clinical Global Impression-Improvement and Severity-of-Illness scales.ConclusionThe MOS Sleep Scale is a reliable, valid tool for assessing changes in the sleep of subjects with moderate-to-severe primary RLS. The somnolence domain failed to relate to clinical severity of RLS, indicating a possible sleep–wake relationship unique to RLS. Use of this scale to evaluate other conditions causing sleep disturbance is supported.     

Validity, sensitivity and specificity of the mentation, behavior and mood subscale of the UPDRS

OBJECTIVE: The unified Parkinson's disease rating scale (UPDRS) is the most widely used tool to rate the severity and the stage of Parkinson's disease (PD). However, the mentation, behavior and mood (MBM) subscale of the UPDRS has received little investigation regarding its validity and sensitivity. Three items of this subscale were compared to criterion tests to examine validity, sensitivity and specificity. METHODS: Ninety-seven patients with idiopathic PD were assessed on the UPDRS. Scores on three items of the MBM subscale, intellectual impairment, thought disorder and depression, were compared to criterion tests, the telephone interview for cognition status (TICS), psychiatric assessment for psychosis and the geriatric depression scale (GDS). Non-parametric tests of association were performed to examine concurrent validity of the MBM items. The sensitivities, specificities and optimal cutoff scores for each MBM item were estimated by receiver operating characteristic (ROC) curve analysis. RESULTS: The MBM items demonstrated low to moderate correlation with the criterion tests, and the sensitivity and specificity were not strong. Even using a score of 1.0 on the items of the MBM demonstrated a sensitivity/specificity of only 0.19/0.48 for intellectual impairment, 0.60/0.72 for thought disorder and 0.61/0.87 for depression. Using a more appropriate cutoff of 2.0 revealed sensitivities of 0.01, 0.38 and 0.13 respectively. DISCUSSION: The MBM subscale items of intellectual impairment, thought disorder and depression are not appropriate for screening or diagnostic purposes. Tools such as the TICS and the GDS should be considered instead.     

Addition of a frequency-weighted score to the Behavioral Pathology in Alzheimer’s Disease Rating Scale: the BEHAVE-AD-FW: methodology and reliability

The Behavioral Pathology in Alzheimer’s Disease Rating Scale (BEHAVE-AD) is a well-established instrument, designed to assess potentially remediable behavioral symptoms in Alzheimer’s disease (AD) patients as well as to evaluate treatment outcome. It consists of 25 symptoms grouped into seven categories. Each symptom is scored on the basis of severity on a four-point scale. A knowledgeable caregiver is queried and items are scored on the basis of symptoms noted in the preceding two weeks. Reliability, construct validity and criterion validity data for the BEHAVE-AD have previously been published. Because of the significance of psychopathology in dementia, it is necessary to optimally describe and define the nature, magnitude and prevalence of behavioral symptomalogy. Accordingly, a frequency component was added to each of the 25 items of the BEHAVE-AD scale. The objective of the present report is to describe this new Behavioral Pathology in Alzheimer’s Disease Frequency-Weighted Severity Scale (BEHAVE-AD-FW) and to establish its inter-rater reliability. In this investigation the BEHAVE-AD-FW scale was administered to caregivers of 28 patients with either mildly impaired cognitive function or a dementia diagnosis. Two clinicians separately and independently rated the responses. Analyses determined that the intraclass correlation coefficients (ICCs) for the frequency component varied between 0.86 and 0.97 for each of the seven BEHAVE-AD categories (p_s < 0.001). ICCs for the frequency-weighted scores (item severity score x item frequency score) ranged from 0.69 to 0.98 for the seven symptom categories (p_s < 0.001). For the BEHAVE-AD-FW total scores, the ICC was 0.91 (P < 0.001). These results indicate that the frequency-weighted component is a reliable addition to the BEHAVE-AD scale.     

The Val30Met familial amyloid polyneuropathy specific Rasch‐built overall disability scale (FAP‐RODS©)

Familial amyloid polyneuropathy (FAP) is a chronic debilitating multi‐organic disorder, mainly assessed using ordinal‐based impairment measures. To date, no outcome measure at the activity and participation level has been constructed in FAP. The current study aimed to design an interval activity/participation scale for FAP through Rasch methodology. A preliminary FAP Rasch‐built overall disability scale (pre‐FAP‐RODS) containing 146 activity/participation items was assessed twice (interval: 2–4 week; test‐retest reliability) in 248 patients with Val30Met FAP examined in Porto, Portugal, of which 65.7% have received liver transplantation. An ordinal‐based 24‐item FAP‐symptoms inventory questionnaire (FAP‐SIQ) was also assessed (validity purposes). The pre‐FAP‐RODS and FAP‐SIQ data were subjected to Rasch analyses. The pre‐FAP‐RODS did not meet model's expectations. On the basis of requirements such as misfit statistics, differential item functioning, and local dependency, items were systematically removed until a final 34‐item FAP‐RODS^© was constructed fulfilling all Rasch requirements. Acceptable reliability/validity scores were demonstrated. In conclusion, the 34‐item FAP‐RODS^© is a disease‐specific interval measure suitable for detecting activity and participation restrictions in patients with FAP. The use of the FAP‐RODS^© is recommended for future international clinical trials in patients with Val30Met FAP determining its responsiveness and its cross‐cultural validation. Its expansion to other forms of FAP should also be focus of future clinical studies.     

Reduced item set for the amyotrophic lateral sclerosis assessment questionnaire: development and validation of the ALSAQ-5

OBJECTIVE: The 40 item amyotrophic lateral sclerosis assessment questionnaire (ALSAQ-40) is a subjective health measure designed specifically to assess areas of importance to patients with ALS. It was designed for use in surveys and clinical trials of this patient group, and has been assessed for reliability and validity. Despite its relative brevity there are situations where an even shorter form of the instrument would be desirable. Consequently, this paper reports a process of item reduction which results in a brief five item version of the instrument. METHODS: Data from two surveys of patients with ALS who completed the ALSAQ-40 were analysed to develop a short form ALSAQ. Questionnaire items were correlated with their dimension total scores. Highly correlated items were transformed onto a scale from 0 to 100 and results compared with the parent dimension. RESULTS: Five items were selected that produced results that closely resembled those of the five dimension scores of the ALSAQ-40. CONCLUSIONS: Results on the new measure compared with the parent ("gold standard") ALSAQ-40 suggest that the measure can produce similar results to the longer form but with considerable economy.     

Assessing autonomic symptoms of Parkinson’s disease with the SCOPA‐AUT: a new perspective from Rasch analysis

Background: The Scale for Outcomes in Parkinson’s disease (PD) for Autonomic Symptoms (SCOPA‐AUT) is a specific scale to assess autonomic dysfunction in PD patients. It was developed and validated under the classic test theory approach. This study sought to test whether the SCOPA‐AUT meets item response theory standards for reliability, internal construct validity, response category ordering, and differential item functioning by gender and age group. Method: The Rasch measurement model was applied to a sample of 385 PD patients. Results: Model fit was obtained after the response categories were rescored and item 10‐Incomplete emptying deleted because of redundancy. Person separation index, a reliability measure, was 0.82. All but two items (2‐Sialorrhea and 13‐Nocturia) were free of gender‐ and age‐related bias. The strict tests of unidimensionality were met, indicating the validity of the total sumscore. Scale targeting suggested the need for items representing milder autonomic symptoms. Conclusions: Suggestions for improving the SCOPA‐AUT include a shorter scale with a simpler response scheme and a combination of sexual items for men and women. The resulting SCOPA‐AUT is a reliable scale, with good internal construct validity, providing Rasch transformed results on a linear metric scale.     

精神疾病住院患者幻觉属性分析

目的：了解幻觉属性的相互关系。方法：应用幻觉量表对83例精神疾病患者进行评定，并进行18项属性的相互的相关分析。结果；18个属性中的大多数属性之间都存在着统计学意义的相关性。结论：幻觉大多数属性相关性好，效应属性项目可能是幻觉的主要特征。     

Reliability and validity of the Chinese version of Narcolepsy Severity Scale in adult patients with narcolepsy type 1

ObjectiveTo evaluate reliability and validity of the Chinese version of Narcolepsy Severity Scale (NSS) in adult patients with narcolepsy type 1 (NT1).MethodsOne hundred and fifty-one adult patients (≥18 years) with NT1 were recruited. All filled out the 15-item Chinese version of NSS. Item analysis included critical ratio and correlation analysis. The validity of NSS was assessed by exploratory factor analysis, discriminant validity and convergent validity. Reliability of NSS was assessed by Cronbach's α coefficient, spilt-half reliability and test-retest reliability.ResultsCritical value of all 15 items ranged from 3.01 to 13.36. Each item was significantly correlated with the total score by a correlation coefficient (r) ranging from 0.219 to 0.700. Three common domains were extracted and 15 items explained 54.86% of the total variance. There was a shift in domains compared to the English version likely due to cultural differences. Cronbach's α coefficient for the total scale of 15 items was 0.821 and for three factors was 0.726, 0.748 and 0.760 respectively. The NSS had good correlation with Epworth sleepiness scale scores, Insomnia severity index scores and moderate correlation with mean the sleep latency of polysomnographic recording, and European Quality of Life-5 Dimensions Questionnaire. The Chinese version of NSS showed good spilt-half reliability and test-retest reliability.ConclusionThe Chinese version of NSS shows satisfactory psychometric properties with good validity and reliability. It is applicable to evaluate the severity and consequences of symptoms in Chinese adult patients with NT1.     

The specificity of depressive symptoms in patients with Alzheimer's disease

OBJECTIVE: This study assessed the specificity of depressive symptoms in patients with Alzheimer's disease and examined the discrepancies between patient and caregiver symptom reports. METHOD: The study group was composed of a series of 233 patients with Alzheimer's disease, 47 patients with depression but without dementia, and 20 healthy comparison subjects; the latter two groups were comparable in age with the patients with Alzheimer's disease. The patients and comparison subjects received a comprehensive psychiatric evaluation, which included administration of the Hamilton Depression Rating Scale and the Structured Clinical Interview for DSM-IV. RESULTS: Patients with Alzheimer's disease with a score of 2 or higher on the "depressed mood" item of the Hamilton depression scale, as scored by their respective caregivers, comprised a group with depressed mood (N=92), whereas patients who scored 0 on this item comprised a group without depressed mood (N=62). A statistical comparison of the scores on the remaining Hamilton depression scale items (2-16) between the Alzheimer's disease patients with and without depressed mood revealed significant differences on all items, except "loss of appetite." However, there were no significant differences on any single Hamilton depression scale item between the Alzheimer's disease patients without depressed mood and the age-comparable healthy comparison subjects. CONCLUSIONS: Depressive symptoms are not widespread among patients with Alzheimer's disease but are significantly related to an underlying depressed mood. Patients with Alzheimer's disease may not be fully aware of the extent of their depressive symptoms.     

Improving fatigue assessment in immune‐mediated neuropathies: the modified Rasch‐built fatigue severity scale

Abstract Fatigue is a major disabling complaint in patients with immune‐mediated neuropathies (IN). The 9‐item fatigue severity scale (FSS) has been used to assess fatigue in these conditions, despite having limitations due to its classic ordinal construct. The aim was to improve fatigue assessment in IN through evaluation of the FSS using a modern clinimetric approach [Rasch unidimensional measurement model (RUMM2020)]. Included were 192 stable patients with Guillain‐Barré syndrome (GBS), chronic inflammatory demyelinating polyradiculoneuropathy (CIDP) or polyneuropathy associated with monoclonal gammopathy of undetermined significance (MGUSP). The obtained FSS data were exposed to RUMM2020 model to investigate whether this scale would meet its expectations. Also, reliability and validity studies were performed. The original FSS did not meet the Rasch model expectations, primarily based on two misfitting items, one of these also showing bias towards the factor ‘walking independent.’ After removing these two items and collapsing the original 7‐point Likert options to 4‐point response categories for the remaining items, we succeeded in constructing a 7‐item Rasch‐built scale that fulfilled all requirements of unidimensionality, linearity, and rating scale model. Good reliability and validity were also obtained for the modified FSS scale. In conclusion, a 7‐item linearly weighted Rasch‐built modified FSS is presented for more proper assessment of fatigue in future studies in patients with immune‐mediated neuropathies.     

The Effect of Failure on Cognitive and Psychological Symptom Validity Tests in Litigants with Symptoms of Post-traumatic Stress Disorder

This study examined the influence of performance on cognitive and psychological symptom validity tests on neuropsychological and psychological test performance in claimants evaluated in a medico-legal context (N =?301) with symptoms of PTSD. A second purpose of this study was to examine the influence of the severity of PTSD symptoms on cognitive test performance after excluding patients who failed to put forth adequate best effort and who exaggerated psychiatric symptoms. Patients were administered a battery of neuropsychological measures that were aggregated into a composite measure, the Cognitive-Test Battery Mean (C-TBM). Patients were also administered a battery of psychological tests that were aggregated into another composite measure, the Psychological-Test Battery Mean (P-TBM). We found that failure on cognitive symptom validity tests was associated with significantly poorer neuropsychological functioning, but there was not a significant effect on psychological symptoms. Conversely, failure on psychological symptom validity tests was associated with higher levels of psychopathology, but there was not a significant effect on cognitive ability. Finally, once patients were screened for adequate effort and genuine symptom reporting, the severity of PTSD symptoms did not appear to influence cognitive ability. This is the first study that assessed both types of symptom validity testing in PTSD claimants, which is important given that previous literature has demonstrated cognitive impairment in PTSD and that individuals with PTSD tend to claim cognitive impairment. Implications of these findings are discussed with regard to the existing literature and the relationship between these two types of symptom validity tests.     

The effect of failure on cognitive and psychological symptom validity tests in litigants with symptoms of post-traumatic stress disorder

This study examined the influence of performance on cognitive and psychological symptom validity tests on neuropsychological and psychological test performance in claimants evaluated in a medico-legal context (N = 301) with symptoms of PTSD. A second purpose of this study was to examine the influence of the severity of PTSD symptoms on cognitive test performance after excluding patients who failed to put forth adequate best effort and who exaggerated psychiatric symptoms. Patients were administered a battery of neuropsychological measures that were aggregated into a composite measure, the Cognitive-Test Battery Mean (C-TBM). Patients were also administered a battery of psychological tests that were aggregated into another composite measure, the Psychological-Test Battery Mean (P-TBM). We found that failure on cognitive symptom validity tests was associated with significantly poorer neuropsychological functioning, but there was not a significant effect on psychological symptoms. Conversely, failure on psychological symptom validity tests was associated with higher levels of psychopathology, but there was not a significant effect on cognitive ability. Finally, once patients were screened for adequate effort and genuine symptom reporting, the severity of PTSD symptoms did not appear to influence cognitive ability. This is the first study that assessed both types of symptom validity testing in PTSD claimants, which is important given that previous literature has demonstrated cognitive impairment in PTSD and that individuals with PTSD tend to claim cognitive impairment. Implications of these findings are discussed with regard to the existing literature and the relationship between these two types of symptom validity tests.