Inter-observer reliability of clinical outcome measures in a lower limb amputee population

Purpose : In an attempt to find a more clinically useful functional outcome measure specifically tailored for lower limb amputees undergoing inpatient prosthetic rehabilitation, a 6-month prospective assessment of inter-rater reliability for Harold Wood - Stanmore Mobility Scale Data, including two handicap scales, was undertaken. An analysis of the data is presented in this paper. Methods : An inter-rater reliability study was undertaken using four observers to complete admission and discharge scores for the three disability/handicap scales on 14 consecutive patients over 6 months. Results : The disability mobility scale demonstrated perfect observer agreement on admission and at discharge the inter-rater reliability for this measure was high (0.83). By contrast, reliability between observers for admission scores on the handicap mobility scale was poor at 0.49 but reasonably high on discharge (0.83). On admission, inter-rater reliability for handicap physical independence was very low (0.15). At discharge, reliability improved to 0.69 being more consistent with results achieved for the other axes. Conclusions : This study confirms the good inter-rater reliability demonstrated previously in the literature but reveals poor inter-rater reliability for the two handicap scales. The latter will require modification before they can be used with confidence in conjunction with the disability scale.     

A new stroke activity scale-results of a reliability study

Purpose : To investigate the internal consistency, inter-rater and intra-rater reliability of a disability stroke activity scale (SAS) for stroke patients. Its intended use is as a measure of motor function at the level of disability in stroke patients.

Method : Twelve stroke in-patients were video-recorded performing the five activities from the SAS. Seven senior physiotherapists, experienced in stroke care, independently rated the recordings on two occasions, three weeks apart, using the SAS. Twelve hospital inpatients participated in the study. The subjects were aged between 48 and 86 and were between 6 and 87 days post stroke.

Results : Reliability for total scores was found to be excellent (generalizability correlation co-efficient (GCC) values ≥0.95) and reliability for individual item scores was good (kappa ≥0.7). Internal consistency reliability using Cronbach's alpha was also good (0.68 at time 1 and 0.68 at time 2).

Conclusion : The stroke activity scale is a reliable instrument for hospital stroke patients. It can be administered in less than 10 minutes and requires minimal equipment and training. Further work on the validity and responsiveness of the SAS is in progress.     

Awareness of disabilities in stroke rehabilitation--a clinical trial

Purpose : To investigate the frequency of unawareness of disabilities after stroke during the rehabilitation stage, the relationship of unawareness with neuroanatomical variables, and the impact of unawareness on functional outcomes.

Method : Sixty consecutive patients (36 with right, 24 with left hemisphere damage) admitted to rehabilitation hospital with a first, single, unilateral stroke were evaluated at admission, discharge and at 1-year post onset of stroke. Unawareness of disabilities was operationally defined as the discrepancy between therapist and patient's rating on the motor scale of the functional independence measure (FIM). Functional outcomes included FIM, instrumental activities of daily living (IADL) scale, activity card sort (ACS) and safety rating scale.

Results : Unawareness of disabilities was found in 44/60 patients at admission and 24/57 at discharge. There was no significant difference between the hemisphere groups in the frequency of unawareness at both times. Discharge unawareness in the right hemisphere group was significantly associated with lesions in the frontal and temporal lobes, and with lesion size. Unawareness in the left hemisphere damaged group was not associated with any neuroanatomical variables. A negative impact of unawareness at admission on functional outcomeswas not found, but it was found that unawareness at discharge was a negative predictor of activity level (ACS score) at follow up, after controlling for the severity of initial disability level.

Conclusions : Unawareness of disabilities is a significant issue in stroke rehabilitation. Unawareness that persists to discharge from rehabilitation correlates with neuroanatomical variables in right hemisphere damaged patients, and is a negative predictor for some rehabilitation outcomes at follow-up.     

Impairments, disabilities and handicap pattern in konzo - a non-progressive spastic para/tetraparesis of acute onset

Purpose: To assess impairments, disabilities and handicap pattern in konzo.

Method: The study included 17 konzo subjects, of which three were males and 14 females (mean age 21, median 18 years). A detailed neurological examination was performed on all subjects. Subsequently, an assessment of impairments, disabilities and handicap was done with a constructed rating scale partially based on the ICIDH-2 framework.

Results: The overall disablement picture in all subjects consisted of motor dysfunction in lower limbs leading to limitations in walking and movement activities, and restrictions in mobility. Hip mobility was severely impaired in most cases (15/17). Although konzo subjects showed normal muscle power in upper limbs (13/17), they had impaired fine motor function (10/17).

Conclusion: Further studies are needed to assess the effectiveness of the WHO criteria for konzo in defining its forms. The applicability of the ICIDH-2 framework in this study demonstrates the possibility of its use as a common language among researchers in the field of motor disorders. However, a revision is suggested of its taxonomy, and a definition of operational criteria to clarify the content of different qualifiers provided to assess the level of functioning or disability.     

Reliability of the Home Falls and Accidents Screening Tool (HOME FAST) for identifying older people at increased risk of falls

Purpose : The Home Falls and Accidents Screening Tool (HOME FAST) was designed as part of a comprehensive health assessment instrument to measure falls risk for older people within their home environment. This paper describes the evaluation of the inter-rater reliability of the HOME FAST.

Method : Forty home visits were undertaken, in urban and rural settings of the UK, by pairs of raters, one of whom was an expert rater. Occupational therapists, occupational therapy assistants and a social worker rated each home using the HOME FAST concurrently with an expert rater. The kappa statistic was used to determine the degree of agreement between pairs of raters.

Results : The overall kappa value for the checklist was 0.62, indicating a fair to good level of agreement between raters. 'Hazardous outside paths' was the only item that demonstrated poor agreement (kappa=0.30). The expert rater consistently identified more hazards than the other raters, and the level of agreement was stable between raters irrespective of the number of hazards present in the homes visited.

Conclusions : Definitions of non-applicable household features and improving the objectivity of the operational definitions for recognizing hazards that demand more qualitative judgements will enhance the reliability of the HOME FAST.     

Development of a measure of receptivity of the physical environment

Purpose. New models of disability identify the importance of measuring the influence of the environment (environmental barriers) on the performance of persons with disabilities. The objective of this paper is to present a new measure of the receptivity of the physical environment for persons with mobility impairments and to offer preliminary information about its psychometric properties.

Methods. The measure, The Community Health Environment Checklist (CHEC), was developed and validated in a community setting with a group of persons with mobility impairments. Sixty-three destinations (buildings, recreational areas or facilities) were assessed using the CHEC.

Results. Using Cronbach's alpha, the CHEC was found to have an internal consistency reliability of 0.95. The content validity of the CHEC was assured by the development procedure.

Conclusion. The CHEC offers a brief, easily administered measure of receptivity of the physical environment for persons with mobility impairments that is psychometrically sound.     

康复护理日常生活活动评定量表的信度和效度研究

背景在康复医学中有许多种日常生活活动(ADL)能力评定量表,但符合康复护理程序的评定量表甚少.为此,浙江省康复中心从2000年开始研究设计了康复护理ADL(rehabilitative nursing ADL,RNADL)评定量表,并制定了评定内容和评定标准.目的探讨康复护理ADL评定量表在患者日常生活活动功能评测中的重测信度、内部一致性及量表的有效信度.设计重复测量设计.单位浙江省望江山疗养院,浙江省康复中心.对象2001-04/2003-04浙江省望江山疗养院康复医学科进行康复治疗的ADL功能障碍的40例患者.方法对40例住院康复患者用RNADL量表进行评测者内部、评测者之间的信度分析;内部一致性用Cronbach α计算;并分别用RNADL和改良的Barthel指数(MBI)进行出、入院各项目评定,用Pearson分析入院、出院及出入院之间的相关性.主要观察指标RNADL评定量表的信度和效度检测结果.结果RNADL评测者内部和评测者之间的信度良好,各项组内相关系数(ICC)范围为0.966～0.998,r=0.996～0.997;内部一致性信度系数α=0.822,各项之间α=0.925,RNADL内部一致性较好.RNADL和MBI的比较研究得出出、入院RNADL与MBI改变值间相关性良好(r=0.89,P＜0.001).结论RNADL具有很高的重测信度,内部一致性较好,与MBI有良好的关联效度,是康复护理日常生活活动能力可信、有效的评测量表.     

Validity and reliability of sincerity test for dynamic trunk motions

Purpose : Marras et al . developed a technique to evaluate sincerity of effort during dynamic trunk motion performance. The validity and reliability of the technique have not been evaluated. Therefore, the objective of this study was to first determine whether or not a sincerity of effort measure correctly identified those giving a sincere effort in a blinded randomized control trial and second to quantify inter-rater and test-retest reliability.

Methods : This article reports the findings of a two phase study. In phase one, the blinded evaluation, participants were randomly assigned to either a sincere or insincere performance condition. An examiner tested participants without knowledge of the participant's group membership. In the second phase, two examiners evaluated each participant twice to quantify inter-rater and test-retest reliability.

Results : In the blinded phase the specificity was 100% and sensitivity was 90% for identifying sincere and insincere effort, respectively. Phase two results showed no significant difference in probability of sincere effort between raters or between testing sessions.

Conclusion : A performance criterion that accurately identifies sincere vs insincere group membership during functional evaluations was identified. There were no significant differences between raters or between testing sessions. These findings indicate that this test is reliable and possesses good predictive validity in assessing sincerity of effort.     

Predictors of handicap situations following post-stroke rehabilitation

Purpose : Many stroke survivors have to cope with impairments and disabilities that may result in the occurrence of handicap situations. The purpose of the study was to explore bio-psycho-social predictors of handicap situations six months after discharge from an intensive rehabilitation programme.

Methods : At discharge from a rehabilitation programme, participants were evaluated with instruments measuring motor, sensory, cognitive, perceptual, affective and psychosocial impairments and disabilities that may play a role in the development of handicap. Some other demographic and clinical variables, and those related to rehabilitation, were also collected. Six months later, they were re-assessed in their own environment in order to document their handicap level with the Assessment of Life Habits (LIFE-H).

Results : One hundred and thirty-two stroke patients participated in the discharge evaluation and 102 of them also participated in the handicap measurement. Relationships between handicap level and impairments and disabilities were all statistically significant. Multiple regression analyses indicated that affect, lower extremity co-ordination, length of stay in rehabilitation, balance, age and comorbidity at the end of an intensive rehabilitation programme are the best predictors of handicap situations six months later (adjusted R 2 : 68.1%).

Conclusions : In spite of its exploratory nature, this study revealed that, among a substantial number of personal characteristics, some were more related to a handicap measure and have greater predictive value. Other studies should be carried out to validate these findings and to consider more environmental factors in order to better understand factors related to the development of handicap situations.     

REPAS, a summary rating scale for resistance to passive movement: item selection, reliability and validity

Purpose. To establish: (i) item characteristics and item selection for the REPAS; (ii) internal consistency, inter-rater and test-retest reliability of the final REPAS version and its subtests; (iii) the association between the REPAS and selected other clinical scales of impairment and activity limitation.

Method. Thirty-three neurological patients with central paresis. Two REPAS assessments with a one-week interval by two independent raters. Concurrent assessment of the Motricity Index, Box-and-Block test, Functional Ambulation Category, Timed walking, Barthel Index, Disability Rating Scale, Carer Burden Scale, and Hygiene Score.

Results. Twenty-six of 52 REPAS items fulfilled the item selection criteria. The final test version showed a high internal consistency, inter-rater and test-retest reliability (correlation coefficients: 0.87 - 0.97, no significant difference between raters or with test repetition). Reliability of the arm and leg subtests was substantial (correlation coefficients: arm subtest 0.63 - 0.98, leg subtest 0.56 - 0.96). REPAS scores were moderately associated with basic ADL competence and a carer's burden with arm or leg adductor spasticity. The REPAS, arm subtest scores, degree of arm paresis and gross manual dexterity showed a moderately high association.

Conclusions. The Ashworth scale-based guidelines assured comparability of test administration and scoring. The REPAS is a reliable and valid summary rating scale for resistance to passive movement.     

The Falls-Efficacy Scale,Swedish version: does it reflect clinically meaningful changes after stroke?

Purpose : The overall aim of this prospective investigation was to evaluate the ability of the Falls Efficacy Scale (Swedish version) (FES(S)) to reflect clinically meaningful changes over time.

Method : Changes on the FES(S) scale were compared with changes in two different standardized measures of observer-assessed balance, the Berg Balance Scale (BBS), the Fugl-Meyer balance subscale (FMB), and of motor function and ambulation in 62 stroke patients. Assessments took place on admission for rehabilitation, at discharge and 10 months after the stroke. Indices of effect size were used to evaluate responsiveness of the instruments. Three time periods were studied: admission to discharge (early response), discharge to 10 month follow-up (late response) and admission to follow-up (overall response).

Results : The main findings are that the FES(S) is as responsive as BBS and FMB in detecting changes during the early and overall response periods. Changes in FES(S) scores between admission and discharge correlated significantly with changes in observer-assessed balance, motor function and ambulation scores.

Conclusions : The present results suggest that measurement of perceived confidence in task performance using the FES(S) scale is responsive to improvement in patients with hemiparesis at an early stage after stroke.     

The QE Health Scale (QEHS): assessment of the clinical reliability and validity of a spiritually based holistic health measure

Purpose. To assess the clinical reliability and validity of a holistic health measure, the QE Health Scale (QEHS), for use with people with physical disabilities.

Method. A test-retest design saw the QEHS administered and compared with established measures of health at admission and discharge from three-week inpatient rehabilitation programmes. Data was analysed by factor and correlation analysis. Clinician-reported credibility and usefulness of the theoretical basis of the QEHS, the QEHS itself, and Patient Profiles derived from the QEHS were also used to evaluate clinical validity.

Results. The QEHS was judged to possess satisfactory reliability and validity.

Conclusion. The QEHS is a clinically reliable, valid, credible and useful holistic health instrument to facilitate client-centred therapeutic interventions, inform decision-making and evaluate outcomes for people with physical disabilities.     

Evaluation of patient education in spinal cord injury rehabilitation: knowledge, problem-solving and perceived importance

Purpose. Through inpatient education programmes the person with spinal cord injury (SCI) learns to understand and monitor his or her own physical, emotional and social well-being. The purpose of this study was to determine the patients' knowledge and problem-solving skills regarding SCI at admission, discharge and follow-up at 6 months after discharge; and to determine the perceived importance of each content topic included in the education programme.

Methods. A one-group repeated measures design was used to evaluate the outcomes. Knowledge was evaluated with a Multiple Choice Questionnaire (MCQ). Problem-solving ability based on participants' responses to Life Situation Scenarios relevant to each topic area was rated on a standardized four-point criterion reference scale. Perceived importance for each topic area was rated on a five-point Likert scale.

Results. Twenty-three participants completed all assessments. There was significant improvement in MCQ scores from admission to discharge (P = 0.04) and admission to follow-up (P = 0.02). For problem-solving ability, there was a trend toward improvement in all content topics with significant improvement from admission to follow-up for the topic of bowel care (P = 0.004). However, many participants continued to demonstrate poor problem-solving ability. Bowel, Bladder and Skin Care were consistently perceived as the most important education topics.

Conclusions. Improvements in knowledge do not necessarily translate to improvements in problem-solving ability even for the topics perceived as important. This may indicate the need to incorporate more active learning strategies or contextually based strategies within patient education programmes to facilitate the transfer of knowledge within life situations.     

Impairment,disability, and life satisfaction in Parkinson's disease

Purpose : To determine the correlation of motor and musculoskeletal impairments with disability scores and life satisfaction. Musculoskeletal impairments might be due to the impact of motor impairments, however some cases may have these impairments even before the disease is diagnosed.

Methods : 48 consecutive patients (32 male, 16 female) with Parkinson's disease with a mean age of 65.6, mean age of onset of the disease of 59.8, and Hoehn and Yahr staging system 1-4 participated. Unified Parkinson's disease rating scale, modified Schwab and England activities of daily living scale and life satisfaction index were used. Also New York modified mobility rating scale for c-spine and upper extremities and trunk and lower extremities was performed.

Results : Motor and musculoskeletal impairments were correlated with the disability score. Bradykinesia, and tremor were correlated with upper and lower extremity range of motion limitation, but rigidity was not. Bradykinesia was correlated with life satisfication. Having had the disease longer was related to upper extremity range of motion limitation.

Conclusions : Motor and musculoskeletal impairments are correlated with disability scores. Bradykinesia and musculoskeletal impairments are correlated with life satisfication. Patients with Parkinson's disease might benefit from a rehabilitation programme focusing on motor impairments such as bradykinesia as well as musculoskeletal impairments even in the early stages of the disease.     

A systematic review of the Tardieu Scale for the measurement of spasticity

Background. The Tardieu Scale has been suggested a more appropriate clinical measure of spasticity than the Ashworth or modified Ashworth Scales. It appears to adhere more closely to Lance's definition of spasticity as it involves assessment of resistance to passive movement at both slow and fast speeds.

Objective. To review the available literature in which the Tardieu Scale has been used or discussed as a measure of spasticity, with a view to determining its validity and reliability.

Study design. A systematic review of all literature found related to the Tardieu Scale (keywords: Tardieu scale, spasticity) from Pubmed and Ovid databases, including medline, CINAHL, EMBASE, Journals at Ovid full text, EBM reviews and Cochrane database of systematic reviews. Hand searching was also used to track the source literature.

Conclusions. In theory, we can acknowledge that the Tardieu Scale does, in fact, adhere more closely to Lance's definition of spasticity. However, there is a dearth of literature investigating validity and reliability of the scale. Some studies have identified the Tardieu Scale to be more sensitive than other measures, to change following treatment with botulinum toxin. Further studies need to be undertaken to clarify the validity and reliability of the scale for a variety of muscle groups in adult neurological patients.     

Masculine role variables and outcomes among men with spinal cord injury

Purpose. Few studies have considered the impact of masculine role variables on outcome and adjustment to SCI among men. The present study examined the relations among SCI, views of masculinity, psychological adjustment, and rehabilitation outcomes among men with SCI.

Method. The sample included 20 men with SCI receiving inpatient rehabilitation, with a mean age of 45 years. Data included demographic variables as well as Conformity to Masculine Norms Inventory (CMNI), Gender Role Conflict Scale (GRCS), Functional Independence Measure (FIM), and Satisfaction with Life Scale ratings, and change in marital status.

Results. The findings revealed that satisfaction with life was positively related to scores on the CMNI Violence scale, FIM change from admission to discharge was positively related to the CMNI Emotional Control scale and negatively related to the CMNI Dominance scale. Change in marital status was inversely related to the CMNI Emotional Control and Primacy of Work scales and the GRCS Restricted Emotionality and Power, Success, and Competition scales.

Conclusions. The findings show that certain aspects of the traditional masculine role (i.e., ability to modulate strong emotions) may be adaptive in the rehabilitation process, whereas other aspects (i.e., a dominant interpersonal style) may present a barrier to effective rehabilitation.     

Reliability, validity and factor structure of the upper limb subscale of the Motor Assessment Scale (UL-MAS) in adults following stroke

Purpose: The upper limb items of the Motor Assessment Scale (MAS) have been shown to be a sensitive, valid and reliable measure of upper limb function for adults following stroke, however the validity and reliability of summing these items into an independent subscale has not yet been evaluated. The stability, internal consistency and construct validity of the upper limb MAS subscale (UL-MAS) was assessed in this study.

Method: Twenty-seven inpatients following stroke (mean age = 67 years, range = 40 - 80) were sampled from an acute, inpatient rehabilitation setting. Patients were evaluated with 'Upper Arm Function', 'Hand Movements', and 'Advanced Hand Activities' items of the MAS by masked physiotherapists who had received standardized training in administration of the MAS.

Results: All items were explained by one factor on confirmatory factor analysis and correlated significantly with one another and with the composite (summed total) score. Internal consistency analysis produced a Cronbach's alpha of 0.83 which did not benefit from removal of any items.

Conclusions: The acceptable internal consistency score obtained verifies the validity and reliability of using the UL-MAS as an independent scale. This study has also verified the construct validity of the UL-MAS subscale and provides a valuable extension of previous work, which together demonstrates the value of the UL-MAS as a responsive, valid and reliable measure of upper limb function in adults following stroke. The UL-MAS produced a single, composite score that could be interpreted as a total score for upper limb function in this population.     

A comparison of upper limb amputees and patients with upper limb injuries using the Disability of the Arm, Shoulder and Hand (DASH)

Purpose: To use the Disability of the Arm Shoulder and Hand (DASH) scale to measure the disability of patients with upper limb amputation(s) and to compare these to other upper limb injuries.

Method: All 274 patients over the age of 18 years presenting to Prince Henry Hospital in Sydney over a 4-year time frame were given the DASH assessment tool and asked to complete it under supervision of the Occupational Therapist.

Results: Patients with brachial plexus injuries, Complex Regional Pain Syndrome and bilateral upper limb amputations demonstrated significantly higher levels of disability to patients with unilateral upper limb amputations. Partial hand amputees reported a higher level of disability than major unilateral upper limb amputees. For the 48 patients who completed pre- and post-treatment assessments, there was a significant improvement in their health status.

Conclusions: Further research is required to understand the factors that affect a patient's perceptions of their disability. Perhaps the definitive nature of an amputation and the immediate involvement of highly skilled health professionals serve to assist patients to accept their injury and therefore minimizes the level of disability.     

The measurement of psychotic acuity by nursing staff

Accurate evaluation of patients' psychotic state is essential to decrease psychotic symptoms and protect the patient and others. The aim of this paper is to conduct a literature review in order to access the utility, reliability and validity of current rating scales that are purported to measure psychotic acuity of inpatient population. A search of a number of electronic databases was undertaken to retrieve potential articles that focus on the measurement of acute psychosis. We identified some conceptual and theoretical problems when using a scale that is monitoring the progress of discharge and assesses the outcome of treatments. The findings revealed a difficulty in finding a commonly agreed definition of acute psychosis and a problem of obtaining frequent measures, and the frequency of measurement and fluctuation in psychosis. The most dominant scales in assessing psychosis were reviewed: the Global Assessment of Functioning Scale, the Brief Psychiatric Rating Scale and the Positive and Negative Symptom Scale. Several issues related to the scales' inter-rater reliability and construct validity remain unexplored. None of these scales addressed the conceptual and theoretical problems that we identified. A new scale that will measure acuity of symptoms in inpatient settings needs to be created.     

Comparison of three preverbal scales for postoperative pain assessment in a diverse pediatric sample

The purposes of this study were to compare the reliability and validity of three pain measurement scales for assessing pain in preverbal and nonverbal children and to determine which of the scales was mos tappropriate in a clinical setting to evaluate pain for infants and young children regardless of developmental stage or cognitive or physical disability. Pain scales tested were revised versions of the Riley Infant Pain Scale (RIPS), the Nursing Assessment of Pain Intensity (NAPI), and the Postoperative Pain Score (POPS). Purposive sampling of 391 postoperative infants and children was used for evaluation of pain in a midwestern children's hospital. Four assessments with each scale were done 1 hr apart by trained observers blinded to pain medications. Data analyses supported high inter-rater reliability, satisfactory discrimination between pain and no-pain observations, and suggested acceptability for all three scales with lower caregiver burden for RIPS and NAPI.