Reliability of the Perceive, Recall, Plan and Perform System of Task Analysis: A criterion-referenced assessment

Objective:  To conduct preliminary examination of the rater and test-reliability of the Perceive, Recall, Plan and Perform (PRPP) System of Task Analysis, an ecological measure designed to assess task-embedded information processing capacity during occupational therapy assessment of confused and agitated adults following traumatic brain injury.
Methods:  Occupational therapists observed and scored client performance using the PRPP System of Task Analysis. Correlational analysis and measures of agreement were performed to determine interrater and intrarater reliability. Test procedures were examined for reliability and internal consistency.
Results:  Interrater and test reliability considered three factors: therapists, clients and tasks. A moderate level of interrater reliability was achieved between trained therapists (intraclass correlation coefficient (ICC) = 0.60). Test procedures were highly reliable (ICC = 0.88). Across two measurement occasions, therapists showed a tendency towards harder rating on the second test occasion (      –4.5%; 95% confidence interval for     : –10.67%→ 3.17%).
Conclusion:  The findings of this study support the use of criterion-referenced tests in the area of occupational performance measurement. Occupational therapists achieved moderate interrater reliability when measuring the performance of adults with brain injury on various activities of daily living. Test procedures were found to be highly reliable in measuring the occupational performance of adults demonstrating confusion and agitation typical to the stage of post-traumatic amnesia following head injury.     

Using the Perceive, Recall, Plan and Perform System to assess cognitive deficits in adults with traumatic brain injury: A case study

One role of occupational therapy in the rehabilitation of people with traumatic brain injury is to determine the impact of cognitive and perceptual deficits on client performance of occupational tasks. Existing methods that can be used to assess cognitive deficits include standardised instruments and observations of everyday task performance. Few of these assessment methods are able to adequately link problems of cognition with problems with self-maintenance, productivity and leisure tasks relative to a client's expected occupational roles. The Perceive, Recall, Plan and Perform (PRPP) System of Task Analysis enables observation of performance of any functional task and allows therapists to identify the cognitive basis of disordered performance. The results of this assessment are then used to establish and evaluate client-centred goals and guide subsequent intervention. This article describes the PRPP System of Task Analysis and outlines how it can be used by occupational therapists involved in the management of adults with traumatic brain injury. Specifically, an in-depth case study illustrates how this innovative model of assessment has effectively been used to promote client learning of self-maintenance skills.     

Interrater reliability of the Social Profile: Assessment of community and psychiatric group participation

Background/Aim:  The Social Profile was developed based on the theoretical concepts of activity group participation skills: Parallel, Associative, Basic and Supportive Cooperative and Mature activity levels. The aim of this study is to assess the interrater reliability of observations in groups of children and adults using the Social Profile.
Methods:  Two community and two hospital populations were used to determine the interrater reliability of the Social Profile as observed by clinicians and graduate students. These groups included preschool children, older adults, general, geriatric and substance abuse psychiatric patients. Observers consisted of clinicians and graduate student pairs.
Results:  Moderately high intraclass correlation coefficient (ICC) scores of interrater reliability were found in separate and combined scores from the Social Profile measurements.
Conclusion:  These ICC reliability rates indicate that the Social Profile interrater scores are adequately high for acceptable measurement of social and activity participation levels in clinical, community and school groups.     

Profile of cognitive problems in schizophrenia and implications for vocational functioning

Aim:  This literature review attempts to profile specific areas of cognition that have shown unique and consistent evidence of dysfunction among people with schizophrenia. In addition, their impact on vocational functioning is illustrated, so as to highlight the importance of managing these cognitive difficulties in vocational rehabilitation.
Methods:  Literature search was carried out on seven key cognitive domains identified by the National Institute of Mental Health in the USA. Their impact on vocational function was also reviewed.
Results:  It is found that attention, declarative and working memory, reasoning, problem-solving and social cognition are areas of impairment that have great impact on vocational functioning. Attention and memory problems affect learning of new work tasks. Executive function is particularly crucial in determining supported and open employment outcomes, as executive dysfunction cannot be easily compensated. Lastly, social cognition plays a major role in determining the success of workplace social exchanges.
Conclusion:  Occupational therapists need to have a good understanding of the profile of cognitive problems among people with schizophrenia, in order to tailor our intervention according to their cognitive strengths and difficulties. Several cognitive remediation strategies and programs have been designed specifically for people with mental illness. Equipping ourselves with skills in conducting such programs will augment our expertise in vocational rehabilitation.     

Reliability of the original and revised versions of the Assessment of Living Skills and Resources

Background/Aim:  The Assessment of Living Skills and Resources (ALSAR) measures instrumental activities of daily living in the older population and has some established reliability estimates. The study aimed to examine the interrater reliability of the original ALSAR in an Australian context and develop revised scoring criteria that were examined in a second interrater reliability study.
Method:  Seven occupational therapists rated a total of 20 clients over 65 years of age, with 10 clients in each study.
Results:  Using the original criteria, the ALSAR had relatively low interrater reliability across all scores (ICC range 0.253–0.756). The revised version yielded improved interrater reliability on all scores (ICC range 0.541–0.896).
Conclusion:  Detailed scoring criteria increased the interrater reliability of the ALSAR, enhancing its usefulness for clinical practice and research.     

Reliability of a composite measure of social inclusion for people with psychiatric disabilities

Aim:  To assess the reliability of the components of a proposed composite measure of social inclusion for people with psychiatric disabilities.
Method:  The interview covered sociodemographics, domain-specific socially valued role functioning, social support, stigma experiences, integration within the immediate psychosocial rehabilitation community, and integration within the wider neighbourhood community. Participants (n ₁  = 28; n₂  = 26) were recruited from a psychosocial rehabilitation setting. The candidate measures were assessed for short-cycle test–retest reliability and internal consistency.
Results:  Several items were identified for removal from the stigma experiences and community integration subscales because of low test–retest reliability or for having not sufficiently contributed to internal consistency.
Conclusions:  The promising test–retest and internal consistency results warrant continuing the psychometric development of the composite measures of social inclusion and its applicability to community residents with psychiatric disabilities.     

The Perceive,Recall, Plan and Perform (PRPP) system for persons with Parkinson’s disease: a psychometric study

Background: The Perceive, Recall, Plan &; Perform (PRPP) system of task analysis might be feasible to evaluate occupational performance and information processing strategies for persons with Parkinson’s disease (PD).

Aim: To evaluate: (1) the random error between raters (inter-rater study), (2) the random error within raters (intra-rater study), and (3) the internal consistency of the PRPP.

Materials and methods: (1) video-recorded performance of meaningful activities of 13 Dutch persons with PD, scored independently by 38 Dutch PRPP trained occupational therapists were included in the analysis. The random error between raters was analyzed with two-way random Intraclass Correlation Coefficients (ICC). (2) Four videos were scored twice by 30 raters (6 week time interval). The random error within raters was analyzed using one-way random ICC’s. (3) Internal consistency study: data of 190 persons with PD were analyzed using Cronbach’s alpha (α).

Results: Inter-rater reliability ranged from slight to moderate (ICC=?0.06–0.43). The mean intra-rater reliability ranged from moderate to almost perfect (ICC=?0.60–0.83). Internal consistency is good (α?=?0.60–0.86).

Conclusion: The limited inter-rater reliability but adequate intra-rater reliability and internal consistency show the feasibility of the PRPP when used for persons with PD. Implications for reliable clinical use are discussed.     

Digital photography as a tool to measure school cafeteria consumption

Background:  Assessing actual consumption of school cafeteria meals presents challenges, given recall problems of children, the cost of direct observation, and the time constraints in the school cafeteria setting. This study assesses the use of digital photography as a technique to measure what elementary-aged students select and actually consume from school cafeteria meals.
Methods:  Before and after still digital photographs were taken of labeled trays for every lunch served to elementary students over 4 lunch periods. Two analysts visually estimated the amount of each item consumed from every tray, and those estimates were compared to evaluate interrater reliability.
Results:  Collection of photographic data was rapid and did not disrupt the busy elementary cafeteria setting. Analysts' estimates of consumption levels of meal components (main and side dishes) were within 10% of each other in 92% of the cases. Only 0.2% of items could not be analyzed due to children playing with food or otherwise obstructed photographs.
Conclusions:  Digital photography offers researchers and school food service personnel a highly accurate and cost-effective tool to measure actual consumption of school cafeteria meals. Data collected through this method can be evaluated by simple counts of servings of produce or other food groups or by more detailed analyses of nutritional composition.     

Satisfaction with Daily Occupations: Construct validity and test–retest reliability of a screening tool for people with mental health disorders

Background/Aim:  In occupational therapy research and in clinical practice there is a need for valid, reliable and easily administered measures. For research and screening purposes, the Satisfaction with Daily Occupations (SDO) instrument was developed. It addresses work, leisure, domestic tasks and self-care and generates a satisfaction and an activity level score. This study investigated its construct validity, internal consistency and test–retest reliability in 55 clients visiting occupational therapists in outpatient mental health care.
Methods:  The SDO was administered on two occasions with a 1-week interval, and data on the Canadian Occupational Performance Measure (COPM), psychosocial functioning and self-rated health were collected on the first of these occasions.
Results:  In contrast to what was expected, the relationships between the SDO scores and the COPM scores were low. The strongest relationship between the SDO satisfaction score and any other variable was to psychosocial functioning. These variables shared 23% of the variance. Internal consistency for the satisfaction score was acceptable, alpha = 0.75 on the second occasion, and the test–retest reliability was good; r _s   = 0.84 for the satisfaction score and r _s   = 0.92 for the activity level.
Conclusions:  The results indicated that the SDO has satisfactory reliability. The instrument targets a specific construct, as compared to the COPM and the measurements of self-rated health and psychosocial functioning. Thus, the SDO and the COPM satisfaction scale seem to assess different phenomena, probably because the SDO has predefined items, not specifically targeting a certain client's problematic occupations, whereas the COPM is based on self-defined problems. The instruments may therefore supplement each other.     

Health behaviours of young, rural residents: A case study

Objective:  To analyse self-reported health behaviours of young people from a rural community and the factors influencing their behaviours.
Methods:  Interviews were conducted with 19 young people, 11 parents and 10 key informants from a small rural Victorian community, asking about teenage health behaviours and the factors influencing these behaviours.
Results:  Young people ate both healthy and unhealthy foods, most participated in physical activity, few smoked and most drank alcohol. The study found that community level factors, including community norms, peers, access issues and geographic isolation, were particularly powerful in shaping health behaviours, especially alcohol consumption. Smoking was influenced by social participation in the community and national media health campaigns. Diet and exercise behaviour were influenced by access and availability, convenience, family, peers and local and non-local cultural influences.
Conclusion and implications:  The rural context, including less access to and choice of facilities and services, lower incomes, lack of transport and local social patterns (including community norms and acceptance), impact significantly on young people's health behaviours. Although national health promotion campaigns are useful aspects of behaviour modification, much greater focus on the role and importance of the local contexts in shaping health decisions of young rural people is required.     

Interrater reliability of the Personal Care Participation Assessment and Resource Tool (PC-PART) in a rehabilitation setting

Background:  The Personal Care Participation Assessment and Resource Tool (PC-PART), formerly the Handicap Assessment and Resource Tool (HART), assesses the domains of clothing, hygiene, nutrition, mobility, safety, residence and supports.
Aim:  To examine the interrater reliability of the PC-PART in a rehabilitation setting.
Methods:  Assessments made by the researcher were compared to the interdisciplinary rehabilitation team. The research and standard assessments occurred within three working days. Raters were blind to each other's scores. Sample participants were a consecutive case-series of rehabilitation clients with varied diagnoses, activity limitations and participation restrictions. Of 66 consecutive patients seen during the a priori determined enrolment period, 25 were included in the study (nine males and 16 females, aged 44–85 years). The remaining 41 patients did not meet the inclusion criteria.
Conclusion:  The PC-PART has good interrater reliability. Clinicians, administrators and researchers can be reassured about this aspect of the validity of the tool.     

Development of EMPRO: A Tool for the Standardized Assessment of Patient-Reported Outcome Measures

Objective:  This study was aimed to develop a tool for the standardized assessment of patient-reported outcomes (PROs) to assist the choice of instruments.
Methods:  An expert panel adapted the eight attributes proposed by the Medical Outcomes Trust as evaluation review criteria, created items to evaluate them, and included a response scale for each item. A pilot test was designed to test the new tool's feasibility and to obtain preliminary information concerning its psychometric properties. The Spanish versions of five measures were selected for assessment: the SF-36 Health Survey, the Nottingham Health Profile, the COOP-WONCA charts, the EuroQol-5D, and the Quality of Life Questionnaire EORTC-QLQ-C30. We assessed the new tool's reliability (Cronbach's alpha and intraclass correlation coefficient [ICC]) and construct validity.
Results:  The new EMPRO (Evaluating the Measurement of Patient-Reported Outcomes) tool has 39 items covering eight key attributes: conceptual and measurement model, reliability, validity, responsiveness, interpretability, burden, alternative modes of administration, and cross-cultural and linguistic adaptations. Internal consistency was high (α = 0.95) as was interrater concordance (ICC: 0.87–0.94). Positive associations consistent with a priori hypotheses were observed between EMPRO attribute scores and the number of articles identified for the measures, the years elapsed since the publication of the first article, and the number of citations.
Conclusion:  A new tool for the standardized assessment of PRO measures is available. It has shown good preliminary reliability and validity and should be a useful aid to investigators who need to choose between alternative measures. Further assessment of the tool is necessary.     

Validity of a Short Functioning Test (FAST) in Brazilian Outpatients with Bipolar Disorder

Objectives:  As the use of functioning outcomes is increasingly being advocated in multinational clinical trials and comparative studies, making available instruments with known validity and reliability in several languages is required. Here we present data on the Portuguese validation of the Functioning Assessment Short Test (FAST), which was explicitly designed to gauge functioning dimensions empirically linked to bipolar disorder.
Methods:  One hundred patients with bipolar disorder and matched controls were assessed with the FAST, which was evaluated regarding discriminant, content and construct validity, concurrent validity with functioning instruments, internal consistency and test–retest reliability.
Results:  The FAST displayed a five-factor structure very similar to its conceptualization, successfully discriminated patient and control groups, and correlated highly with other functioning measures; it also showed excellent test–retest reliability and internal consistency.
Conclusions:  The FAST is a measure with sufficient validity and reliability, with potential for the use in international clinical trials and comparative studies.     

Validation of the General Nutrition Knowledge Questionnaire in an Australian community sample

Aim:  To validate the General Nutrition Knowledge Questionnaire developed by Parmenter and Wardle (1999) in an Australian community sample. This questionnaire differs from previous assessments of knowledge because it incorporates a broad range of nutrition concepts, including knowledge of dietary recommendations, healthy food choices, nutrient sources and some diet–disease relationships.
Methods:  The original questionnaire was developed in the UK, and thus modified to suit the current Dietary Guidelines for Australians and current public health nutrition recommendations. A total of 156 people, of which 116 were community members, completed the questionnaire (113 items). As an indication of concurrent validity, a subsample of nutrition and dietetic students were included (n = 40). As a measure of test–retest reliability, a subsample (n = 57), including students and community members, answered the questionnaire on two occasions, two weeks apart.
Results:  Both overall internal reliability of the questionnaire items (Cronbach's alpha = 0.92) and test–retest reliability (r = 0.87) were high. The nutrition and dietetic students, hypothesised to have higher knowledge levels, scored consistently higher than the general community sample, indicating good concurrent validity.
Conclusion:  A test of a modified version of the General Nutrition Knowledge Questionnaire found it to be a valid and reliable measure of nutrition knowledge, appropriate for use in a section of the Australian community. The validated tool may be used in the future for the comprehensive assessment of general nutrition knowledge; however, further testing in differing sections of Australian society may be warranted.     

The Assessment of Motor and Process Skills as a measure of ADL ability in schizophrenia

Background: Functional impairments in schizophrenia are substantial, complex, and persistent. Objective measurement of ADL ability, functional capacity and performance is needed for effective intervention planning and outcome evaluation. Objective: To evaluate ADL ability in people with schizophrenia using the Assessment of Motor and Process Skills (AMPS) and to determine the utility of using the AMPS to predict levels of assistance required for successful community living. Method: In a retrospective audit, AMPS ADL measures of a consecutive sample of 64 people with schizophrenia admitted to a mental health facility were compared with normative data and with recommended “cut-off” measures for competency to live independently in the community. Results: Substantial difficulties were measured in both ADL motor (mean z = –1.5) and ADL process ability (mean z = –2.1). AMPS ability measures did not predict problems with independent living for 62.5% of the patients. Conclusion: People with schizophrenia admitted to an inpatient rehabilitation facility experienced significant difficulty performing ADL tasks. AMPS is a useful measure of ADL ability but should be used in conjunction with measures of functional performance in order to plan interventions and supports for people with schizophrenia that reflect the complexity of factors affecting community functioning.     

Validity and reliability of the MEDYN questionnaire for evaluation of functioning in mental health clients receiving occupational therapy

Background:  A number of questionnaires are currently available for measuring functionality of mental health clients; however, they are complex, clumsy to use and impossible to compare to each other. There is a clear need for a simple and straightforward tool that uses quantitative measures. We formulated a questionnaire based on occupational, behavioural and cognitive theories to assess the functioning of psychiatric inpatients receiving occupational therapy.
Methods:  Thirty-one patients hospitalised in a closed and an open ward and receiving occupational therapy were monitored during 4 weeks. Clinicians completed the questionnaire for each patient on a weekly basis.
Results:  Interrater reliability was found to be generally high with correlations among the raters in the closed ward being higher than among those in the open ward. Both cognitive and general/social functioning improved over time and the improvement was associated with type of illness and duration of treatment.
Conclusions:  The MEDYN questionnaire is a valid and reliable objective tool for functional assessment of this subgroup of patients.