Comments on the Reiss Screen for Maladaptive Behaviour and its factor structure

Sturmey's factor analysis of the Reiss Screen for Maladaptive Behaviour is based upon several significant errors, some of which are of general interest because misuse of factor analysis is common (Ford et al. 1986). In this commentary, the author discusses: the concept of factor robustness; appropriate samples for evaluating the factor structure of a dual diagnosis instrument; the differences between exploratory and confirmatory analyses; and how the Reiss Screen was intended to fulfill its purpose. In the Sturmey study, the Reiss Screen may have actually performed well under adverse conditions. The criteria for conducting valid factor analytic studies of dual diagnosis instruments are identified in order to encourage future research evaluating the Reiss Screen and other instruments.     

The French version of the Reiss Screen for Maladaptive Behavior: Factor structure,point prevalence and associated factors

The main aims of the present study were to examine the factor structure and the internal consistency of the factors in the French version of the Reiss Screen of Maladaptive Behavior in a French-speaking European sample. The prevalence of psychopathology and the influence of associated factors were also examined. The Reiss Screen was administered to 467 adults (age range: 18–73) with intellectual disability living in the French-speaking regions of Switzerland and Belgium. A confirmatory factor analysis was performed to replicate the original factor structure. Internal consistency was examined by using Cronbach's alpha. Analyses of variance were computed to study the influence of gender, age and Down syndrome etiology. The original factor structure of the Reiss Screen was replicated. The overall rate of psychopathology in the sample was 37%. No linear relationship between age and psychopathology was found. However, adults aged less than 26 years had lower scores than older adults on several psychopathological domains. Males had higher scores than females on the Autism and the Avoidant Disorder subscales. Participants with Down syndrome had lower scores on all domains, with the exception of the Autism subscale. The results of this study suggest that the French version of the Reiss Screen can be a useful tool to detect psychopathology in adults with intellectual disability.     

A psychometric evaluation of a Swedish version of the psychopathology inventory for mentally retarded adults (PIMRA)

Given the difficulties with symptom identification and the assessment of mental health problems in persons with intellectual disabilities (ID) there has been a focus on the development of relevant assessment schedules for persons with ID. A Swedish version of the psychopathology inventory for mentally retarded adults (PIMRA, informant version), an instrument designed for identification of specific mental health problems in persons with ID, was evaluated in terms of inter-rater reliability, internal consistency, item grouping, criterion and concurrent validity based on a sample of 101 adults with different levels of ID. The Swedish version of the PIMRA had low inter-rater agreement on specific items but good agreement on low or high total score. In order to investigate internal validity, principal component analyses (PCA) were conducted in steps. Consequently, the majority of the 26 items, which remained after three PCA analyses, were grouped in a pattern approximating five of the original PIMRA subscales. The present analysis identified no item grouping matching the original Affective disorder subscale. Based on these results a revised Swedish version was developed. The concurrent validity analysis yielded a strong association between the total scores of the PIMRA and Reiss Screen for Maladaptive Behaviour (RSMB), between subscales on both instruments describing psychotic symptoms and between the RSMB subscale Aggressive behaviour and the PIMRA subscale Adjustment problems. The outcome of the criterion validity analysis indicated that individuals with a clinical diagnosis obtained higher total PIMRA scores than individuals without a clinical diagnosis and a comparison between the PIMRA subscale Psychosis and the clinical diagnosis indicating psychoses according to DSM-III-R or DSM-IV yielded a higher specificity than sensitivity measure. The results indicate that the PIMRA had a potential to identify individuals with mental health problems in persons with different levels (mild, moderate and severe) of ID and identify individuals with a specific mental disorder. Thus, professionals might use the PIMRA as a complement in the diagnostic process.     

The development of a valid and reliable scale for rating anxiety in dementia (RAID)

A rating scale to measure anxiety in dementia sufferers was developed and evaluated in a sample of 51 inpatients and 32 day-hospital patients. Anxiety scores were not related to sex, age, accommodation or DSM-IV diagnosis of the type of dementia. However, both subjects with physical illnesses and subjects with insight into their memory problems had significantly higher anxiety scores.The kappa values for inter-rater reliability ranged from 0.51 to 1 and for test-retest reliability from 0.53 to 1, which indicates moderate to good reliability.The overall agreement on individual items ranged from 82-100% (inter-rater) and 84-100% (test-retest).The professionals working in the care of the elderly and carer groups felt that the scale was comprehensive and all the items in the scale were important, thereby confirming that it has good content validity. The scale significantly correlated with other anxiety scales and also with independent ratings both by a consultant psychiatrist and also nursing staff, indicating good concurrent validity. Anxiety scores were significantly higher in dementia patients who fulfilled modified DSM-IV criteria for anxiety and clinical diagnosis of anxiety disorder.This showed evidence of good criterion validity. Factor analysis showed five factors, including all items of the scale. Scores of 11 and above on the scale indicated significant clinical anxiety. Overall, the scale had good reliability and validity. It should be a useful clinical and research instrument for assessing anxiety in dementia sufferers.     

Evaluation of screening tools for dementia in older adults with mental retardation

We compared groups with and without diagnosed dementia matched on IQ, age, and presence of Down syndrome. The Dementia Scale for Down Syndrome and Dementia Questionnaire for Mentally Retarded Persons were used to assess participants. We developed two performance tasks to determine whether they were useful in separating subjects with and without dementia and also used the Reiss Screen. Both dementia scales and both performance tasks discriminated between groups. The dementia scales were not related to premorbid IQ, age, or gender, whereas performance tasks were related to dementia and IQ but not age or gender. Various Reiss Screen subscales also discriminated between groups. Subscales of the screening instruments and performance tasks were significantly related, indicating congruent validity. Logistic regression was conducted to assess which combination of tests discriminated best between groups.     

Composite versus multiple-rating scales in the assessment of psychopathology in people with mental retardation

This study tested the hypothesis that separate ratings of frequency and duration add significant information to ratings of problem severity in the assessment of psycho-pathology In people with mental retardation. The Reiss Screen was modified to require ratings of problem severity, frequency and duration for each of 38 maladaptive behaviours. The 171 adolescents and adults were rated by caretakers and supervisory staff on the modified Reiss Screen. The three ratings were found to be very highly intercorrelated. Moreover, multiple linear regression analyses revealed that the ratings of frequency and duration did not add significant information to that provided by ratings of problem severity in the prediction of psychiatric diagnosis in the person's case file. The results support the use of composite scaling over multiple-ratings in the assessment of psycho-pathology in persons with mental retardation.     

Moroccan colloquial Arabic version of the Mini International Neuropsychiatric Interview (MINI): qualitative and quantitative validation.

The validation of mini international neuropsychiatric interview (MINI) into Moroccan Colloquial Arabic language demonstrated good psychometric properties. The concordance between translated MINI's and expert diagnoses was good with kappa values greater than 0.80. The reliability inter-rater and test-retest were excellent with kappa values above 0.80 and 0.90, respectively.     

Reliability of the Hamilton Rating Scale for Depression: a meta-analysis over a period of 49 years

The aim of this study was to provide a comprehensive meta-analytic review of the reliability of the Hamilton Rating Scale for Depression (HRSD) for the period 1960-2008, taking into consideration all three types of reliability: internal consistency, inter-rater, and test-retest reliability. This is the first such meta-analytic study of a clinician-administered psychiatric scale. A thorough literature search was conducted using MEDLINE and PsycINFO. The total number of collected articles was 5548, of which 409 reported one or more reliability coefficients. The effect size was obtained by the z-transformation of reliability coefficients. The meta-analysis was performed separately for internal consistency, inter-rater and test-retest reliability. A pooled mean for alpha coefficient in random effects model was 0.789 (95%CI 0.766-0.810). The meta-regression analysis revealed that higher alpha coefficients were associated with higher variability of the HRSD total scores. With regard to inter-rater reliability, pooled means in random effects model were 0.937 (95%CI 0.914-0.954) for the intraclass correlation coefficient, 0.81 (95%CI 0.72-0.88) for the kappa coefficient, 0.94 (95%CI 0.90-0.97) for the Pearson correlation coefficient, and 0.91 (95%CI 0.78-0.96) for the Spearman rank correlation coefficient. A meta-regression analysis showed positive association between inter-rater reliability and publication year. Test-retest reliability of HRSD ranged between 0.65 and 0.98 and generally decreased with extending the interval between two measurements (Spearman r between the duration of interval and test-retest reliability figures = -0.74). Results suggest that HRSD provides a reliable assessment of depression. Figures indicate good overall levels of internal consistency, inter-rater and test-retest reliability, but some HRSD items (e.g., “loss of insight”) do not appear to possess a satisfactory reliability.     

Diagnostic interview for genetic studies (DIGS): inter-rater and test-retest reliability of the French version

The National Institute of Mental Health developed the semi-structured Diagnostic Interview for Genetic Studies (DIGS) for the assessment of major mood and psychotic disorders and their spectrum conditions. The DIGS was translated into French in a collaborative effort of investigators from sites in France and Switzerland. Inter-rater and test-retest reliability of the French version have been established in a clinical sample in Lausanne. Excellent inter-rater reliability was found for schizophrenia, bipolar disorder, major depression, and unipolar schizoaffective disorder while fair inter-rater reliability was demonstrated for bipolar schizoaffective disorder. Using a six-week test-retest interval, reliability for all diagnoses was found to be fair to good with the exception of bipolar schizoaffective disorder. The lower test-retest reliability was the result of a relatively long test-retest interval that favored incomplete symptom recall. In order to increase reliability for lifetime diagnoses in persons not currently affected, best-estimate procedures using additional sources of diagnostic information such as medical records and reports from relatives should supplement DIGS information in family-genetic studies. Within such a procedure, the DIGS appears to be a useful part of data collection for genetic studies on major mood disorders and schizophrenia in French-speaking populations. Received: 23 March 1998 / Accepted: 12 March 1999     

Emotion recognition as a function of social connpetence and depressed mood in Individuals with intellectual disability

The present study was designed to test whether social competence and mood were predictive of the performance by adults with mild or moderate intellectual disability on a matching-to-sample task using facially expressed emotions as stimuli. Thirty-eight subjects were assigned to either a depressed mood group or a non-depressed mood group based on their scores on the two depression sub-scales of the Reiss Screen for Maladaptive Behavior. The groups were matched on sex, age and level of intellectual disability. Each group consisted of 10 women and nine men; 12 participants in each group had mild and seven had moderate intellectual disability, respectively. Social competence was assessed with the Social Performance Survey Schedule (SPSS). Performance on the matching-to-sample task correlated positively with the subjects′ level of intellectual disability, their mood scores and the scores on the Appropriate Skills sub-scale of the SPSS. The implications of these findings for social skills training programmes and limitations of this study are discussed.     

The Crichton Visual Analogue Scale for the assessment of behaviour in the elderly

A visual analogue scale has been constructed to allow relatives and professionals to rate the behaviour of elderly patients. The scale has been shown to have good inter-rater reliability, test-retest reliability and validity. It is suggested that the scale can provide a quick and convenient initial assessment of the patient either at home or in hospital, and possibly also a measure of change.     

A reliability study of the Spanish version of the Social Behaviour Schedule (SBS) in a population of adults with learning disabilities

The reliability of the Spanish version of the Social Behaviour Schedule (SBS) was tested in a vocational setting on a sample of 64 subjects with learning disabilities. Test-retest assessment showed a good percentage of agreement (80%) and adequate kappa values for most SBS items. The overall percentage of agreement of inter-racer reliability was 85% and kappa values were moderate to nearly perfect for 52% of items. Inter-informant analyses produced poorer results, with an average agreement of 43% and inadequate kappa values on 42% of items. The intraclass correlation coefficient (ICC) was 0.64 for test-retest, 0.76 for inter-rarer assessment and 0.94 for inter-informant assessment. The Spearman correlation coefficient was adequate on the test-retest and inter-rater analyses, but not on inter-informant analysis. This low inter-informant agreement could be attributed to environmental factors which alter the reliability of reports from different informants in community settings with high levels of normalization. In such environments, an interview with a key informant may not suffice, and both a careful review of the clinical record and a direct interview with subjects may enhance the reliability of the information attained.     

Measurement of participation in myotonic dystrophy: reliability of the LIFE-H

Person-perceived social participation is an important aspect to measure in rehabilitation in neuromuscular disorders. The objective was to document the test-retest and inter-rater reliability of the Assessment of Life Habits (LIFE-H), in a sample of individuals with myotonic dystrophy (DM1). Twenty-eight participants with myotonic dystrophy aged between 29 and 75 years (mean: 52.7) were recruited. The LIFE-H questionnaire was administered at three different occasions. The LIFE-H demonstrates high test-retest and inter-rater reliability (ICCs: 0.80-0.92) for the total score and subscores (daily activities and social roles). Moderate to high test-retest reliability (ICCs: 0.76-0.92) was found for most categories (8/10) of the LIFE-H. Similar results were obtained for inter-rater reliability (ICCs: 0.68-0.93). Moderate to high agreement using the Bland and Altman method was obtained for most categories. The LIFE-H, a measure of person-perceived social participation, demonstrates adequate test-retest and inter-rater reliability for clinical and research purposes in myotonic dystrophy.     

The psychometric properties of the Social Training and Achievement Record: II. Reliabilities and concurrent validities

ABSTRACT. Three studies are reported which investigated the test-retest and inter-rater reliabilities of the Social Training and Achievement Record as well as concurrent validities with intelligence. Reliabilities of total and sub-scale scores as well as individual items are reported. Test-retest reliabilities over the period of one month were good for total scores, subscale scores and individual items. Inter-rater reliabilities were more modest. At times, inter-rater reliabilities of a small proportion of items were only at chance levels. Concurrent validities with Raven's Matrices and the Weschler Adult Intelligence Scale were comparable with other measures of adaptive behaviour. The role of direct observation in the assessment of adaptive behaviour, in particular with respect to individual skills being selected for teaching, is discussed.     

Caregiving burden of families looking after persons with intellectual disability and behavioural or psychiatric problems

Background During the last decade, there has been a growing interest in outpatient support services for persons with intellectual disability (ID) and psychiatric and/or behavioural problems and their families. In this study, we explore the family burden that is associated with children or adults with ID and behavioural or psychiatric problems living at home. Method The research group consisted of 66 clients of In Casa, a project of outpatient treatment. Their average age is 18.97 years and their average IQ 55.37. The Reiss Screen for Maladaptive Behaviour (adults) or Reiss Scales for Children's Dual Diagnosis (children) and the Nijmegen Family Situation Questionnaire have been administered. Results About 62% of the children and 86% of the adults living at home and referred to In Casa have severe psychiatric or behavioural problems as measured by the Reiss scales. The dual diagnosis has a more negative impact on the family situation than the condition of ID only. Conclusion Parents consider the psychiatric or behavioural problems of their child to be an extra burden and feel it more difficult to raise and manage such a child. This impels them to change the situation and to call on the help of external services. Some methodological questions and the implications of the findings in terms of support needs are discussed.     

Reliability of the Child Behavior Checklist for the assessment of behavioral problems of children and youth with mild mental retardation

The assessment of psychopathology in persons with mental retardation requires reliable and valid instruments. In the present study, the reliability of the Child Behavior Checklist was determined, using data of 42 children and youth with mild mental retardation, with ages from 10 to 18 years. Kappa coefficients and intra-class correlations were computed to determine the reliability at item level and syndrome level. At item level, mean kappa's for inter-rater and test-retest reliability were 0.267 and 0.52, respectively. At syndrome level, mean intra-class correlations for inter-rater and test-retest reliability were 0.493 and 0.775, respectively. These results suggest that the Child Behavior Checklist may not always represent a reliable checklist for the assessment of psychopathology among children and youth with mild mental retardation.     

The Need of Support and Service Questionnaire (NSSQ): a brief scale to assess needs in clients with long-term mental disabilities

We present the Need of Support and Service Questionnaire (NSSQ), a new instrument developed to provide professionals in psychiatric care and community social services with a tool to assess needs of support and service in mentally disabled clients. The final version of the instrument was designed after comments from professionals in psychiatric care, community social services, and the clients and their relatives. A reliability study (test-retest, n=77), inter-rater (n=69) and a validity study (n=529) were performed. In the validity study, comparisons were done with the Camberwell Assessment of Need (CAN) scale, the Global Assessment of Functioning (GAF) scale and the Social and Occupational Functioning Assessment Scale (SOFAS). The test-retest and inter-rater reliability were 0.86 and 0.76, respectively. The percentage of agreement on the individual items ranged from 83.1% to 100.0% (test-retest) and 76.8% to 100.0% (inter-rater reliability). The corresponding kappa coefficients ranged from 0.63 to 1.00 (test-retest) and 0.33 to 1.00 (inter-rater reliability). The comparison between NSSQ and CAN items demonstrated high sensitivity and specificity as well as satisfactory likelihood ratios (LRs). The correlations between GAF and SOFAS scores and the number of needs of support, number of needs of service and the total number of needs of support and service were -0.49, -0.32 and -0.47, and -0.48, -0.26 and -0.45, respectively. All correlations were statistically significant (P<0.01). The results demonstrated that the psychometric characteristics of the NSSQ were satisfactory. Moreover, professionals without any formal training easily used it.     

Psychometric properties and normative data of the French Developmental Behavior Checklist – Adult version

The main aim of this study was to generate psychometric data for the French translation of the Developmental Behavior Checklist – Adult version (DBC-A). In addition, the score distributions were examined by using components of mean-based scores. The DBC-A was administered to 580 adults with intellectual disability recruited in the French-speaking regions of Switzerland and Belgium. A confirmatory factor analysis was performed to evaluate the adjustment of the collected data to the original six-factor model; the internal consistency of the factors was analyzed with Cronbach's alpha. The Reiss Screen for Maladaptive Behavior was used to evaluate the concurrent validity of the instrument. Psychopathology prevalence was also explored. The original factor structure of the DBC-A was replicated, with the internal consistency of the factors ranging from respectable to very good according to the alpha values. When the “screening” cut-off of the DBC-A was used, the DBC-A highlighted a psychopathology prevalence of 38%, which was close to the frequency provided by the Reiss Screen (36%). These results show the robustness of the DBC-A's psychometric properties. Thus, we recommend the use of this instrument to assess and screen psychopathology among French-speaking adults with intellectual disability.     

Psychometric evaluation of a Swedish version of the Reiss Screen for Maladaptive Behavior

Background People with intellectual disability (ID) are afflicted by mental health problems to a greater extent than other individuals, and the coexistence of ID and mental health problems involves diagnostic as well as treatment difficulties. Methods A Swedish version of the Reiss Screen for Maladaptive Behavior (RSMB), an instrument used for identification of mental health problems in people with intellectual disability (ID) was evaluated in terms of inter‐rater agreement, internal consistency, item grouping and criterion validity based on a random sample and a clinical group of adults with ID. Results The Swedish version of the RSMB had moderate‐to‐low inter‐rater agreement on specific items and good internal consistency. The total score was considered to be a fairly reliable measure of a positive or negative result on the RSMB. A principal component analysis yielded seven interpretable components. A close resemblance in sets of items between the Swedish version and the original version of RSMB was found for three subscales: Aggressive Behaviour, Avoidance Disorder and Depression (Behavioural Signs). The outcome of the criterion validity analysis indicated a higher rate of false negatives than false positives. Conclusions The potential influence of concurrent psychopharmacological treatment is discussed. It is concluded that the Swedish version of the RSMB can be used as intended by staff as a primary screening device for the identification of mental health problems among people with ID in a Swedish setting.     

Psychometric evaluation of a Dutch version of the Mini PAS‐ADD for assessing psychiatric disorders in adults with different levels of intellectual disability

Background People with intellectual disabilities (ID) have an increased vulnerability to develop psychiatric problems. Moreover, the early recognition and the accurate diagnosis of psychiatric disorders in the population of persons with ID are challenging. Method A Dutch version of the Mini PAS‐ADD, which is a screening instrument for identification of mental health problems in people with ID, was evaluated in terms of internal consistency, interinformant reliability, item grouping and criterion validity based on a large‐scale random sample (n = 377) and a clinical sample (n = 99) of adults with ID. Results The Dutch version of the Mini PAS‐ADD showed moderate internal consistency, and moderate concordance among informants. Both aspects of the reliability were comparable for different levels of ID. A factor analysis largely confirmed the scale structure. Concurrent validity with the Reiss Screen for Maladaptive Behavior was high for the Depression, Psychosis and Autism scale. The outcome of the criterion‐validity analysis indicated high specificity. The sensitivity for specific psychiatric disorders by the corresponding scales was moderate, but the general sensitivity for the presence of psychopathology on the basis of any of the scales was satisfying. Conclusions The present research reconfirmed the use of the Mini PAS‐ADD as a primary screening device for the identification of mental health problems among people with ID.