A study of reliability and burden of home health assessment using OASIS

The Outcome and Assessment Information Set (OASIS) is used for outcome reporting, quality improvement, and case mix adjustment of per-episode payment for home health care. The research described here addresses interrater reliability of OASIS items and compares clinician time required to complete patient assessment with and without OASIS. Interrater reliability for OASIS data items was estimated using independent assessments by two clinicians for a sample of 66 patients. Incremental assessment time due to OASIS was estimated using interview data from two agency-matched groups of clinical care providers--one group who used OASIS in the assessment and a second group whose assessment did not include OASIS items. Interrater reliability is excellent (kappa > .80) for many OASIS items and substantial (kappa > 0.60) for most items. The reported time required to complete an assessment with OASIS did not differ from the time required for a comparable assessment without OASIS. The results of this study are being used to guide developmental efforts to improve OASIS items. They can also be informative to home health care agencies when interpreting OASIS-based outcome and case mix reports.     

Validation of a generic measure of continuity of care: when patients encounter several clinicians

PURPOSE Patients who regularly see more than one clinician for health problems risk discontinuity and fragmented care. Our objective was to develop and validate a generic measure of management continuity from the patient perspective. METHODS Themes from 33 qualitative studies of patient experience with care from various clinicians were matched to existing instruments to identify potential measures and measurement gaps. Adapted and new items were tested cognitively, and the instrument was administered to 376 adult patients consulting in primary care for a variety of health conditions but seeing clinicians in a variety of settings. After initial psychometric analysis, the instrument was modified slightly and readministered after 6 months. The analysis identified reliable subscales and their association with indicators of continuity. RESULTS Observed factors correspond to 8 intended constructs, with good reliability. Three subscales (12 items) relate to the principal clinician and cover management and relational continuity. Four subscales (13 items) are related to multiple clinicians and address team relational continuity and problems with coordination and gaps in information transfer. Two (11 items) pertain to the patient's partnership in care. Subscales correlate well and in expected directions with indicators of discontinuity (wanting to change clinicians, suffering, and sense of being abandoned, medical errors) and degree of care organization. CONCLUSION The instrument reliably assesses both positive and negative dimensions of continuity of care across the entire system, and the subscales correlate with continuity effects. It supports patient-centered and relationship-based care and can be used as a whole or in part to assess coordination and continuity in primary care.     

A Study of Reliability and Burden of Home Health Assessment Using OASIS

ABSTRACT

The Outcome and Assessment Information Set (OASIS) is used for outcome reporting, quality improvement, and case mix adjustment of per-episode payment for home health care. The research described here addresses interrater reliability of OASIS items and compares clinician time required to complete patient assessment with and without OASIS. Interrater reliability for OASIS data items was estimated using independent assessments by two clinicians for a sample of 66 patients. Incremental assessment time due to OASIS was estimated using interview data from two agency-matched groups of clinical care providers-one group who used OASIS in the assessment and a second group whose assessment did not include OASIS items. Interrater reliability is excellent (kappa > .80) for many OASIS items and substantial (kappa > 0.60) for most items. The reported time required to complete an assessment with OASIS did not differ from the time required for a comparable assessment without OASIS. The results of this study are being used to guide developmental efforts to improve OASIS items. They can also be informative to home health care agencies when interpreting OASIS-based outcome and case mix reports.     

Characterizing Standardized Functional Data at Inpatient Rehabilitation Facilities

ObjectiveThe Improving Medicare Post-Acute Care Transformation Act of 2014 mandates using standardized patient functional data across post-acute settings. This study characterized similarities and differences in clinician-observed scores of self-care and transfer items for the standardized section GG functional domain and the functional independent measure (FIM) at inpatient rehabilitation facilities.DesignWe conducted secondary analyses of 2017 Uniform Data System for Medical Rehabilitation national data. Patients were assessed by clinicians on both section GG and FIM at admission and discharge. We identified 7 self-care items and 6 transfer items in section GG conceptually equivalent with FIM. Clinician-assessed scores for each pair of items were examined using score distributions, Bland-Altman plot, correlation (Pearson coefficients), and agreement (kappa and weighted kappa) analyses.Setting and ParticipantsIn all, 408,491 patients were admitted to Uniform Data System for Medical Rehabilitation-affiliated inpatient rehabilitation facilities with one of the following impairments: stroke, brain dysfunction, neurologic condition, orthopedic disorders, and debility.MeasuresSection GG and FIM.ResultsPatients were scored as more functionally independent in section GG compared with FIM, but change score distributions and score orders within impairment groups were similar. Total scores in section GG had strong positive correlations (self-care: r = 0.87 and 0.95; transfer: r = 0.82 and 0.90 at admission and discharge, respectively) with total FIM scores. Weak to moderate ranking agreements with total FIM scores were observed (self-care: kappa = 0.49 and 0.60; transfers: kappa = 0.43 and 0.52 at admission and discharge, respectively). Lower agreements were observed for less able patients at admission and for higher ability patients of their change scores.Conclusions and ImplicationsOverall, response patterns were similar in section GG and FIM across impairments. However, variations exist in score distributions and ranking agreement. Future research should examine the use of GG codes to maintain effective care, outcomes, and unbiased reimbursement across post-acute settings.     

Inter-rater reliability of the original and modified Barthel Index,and a comparison with the Functional Independence Measure

The Barthel Index (BI), the Modified Barthel Index (MBI) and the Functional Independence Measure (FIM) are all widely used by occupational therapists as assessment tools for clinical decision-making and outcome measurement. All of these tools have demonstrated validity and the BI and the FIM have demonstrated inter-rater reliability. The MBI has been modified to increase sensitivity; however, there have been no publications on the inter-rater reliability of this tool following the changes. The purpose of this research was to examine the inter-rater reliability of two versions of the Barthel Index, and draw some comparisons between this assessment tool and the FIM. Twenty-five patients with neurological and orthopaedic conditions were assessed by three occupational therapists using the three tools. The method of analysis selected was percentage agreement and intraclass correlation coefficient. The results indicated that both the original and modified versions of the Barthel Index possess good inter-rater reliability. As all three tools have demonstrated adequate reliability and validity, it is suggested that clinicians select the most sensitive tool that best meets their clinical needs, and use this assessment tool in its standardized format.     

Gastroparesis Cardinal Symptom Index (GCSI): Development and validation of a patient reported assessment of severity of gastroparesis symptoms

BACKGROUND: Patient-rated symptom assessments are needed for evaluating the effectiveness of medical treatments and for monitoring outcomes in gastroparesis. OBJECTIVE: This paper summarizes the development and psychometric evaluation of a new instrument, the Gastroparesis Cardinal Symptom Index (GCSI), for assessing severity of symptoms associated with gastroparesis. METHODS: The GCSI was based on reviews of the medical literature, patient focus groups, and interviews with clinicians. A sample of 169 patients with a documented diagnosis of gastroparesis participated in the psychometric evaluation study. Patients completed the GCSI, the SF-36 Health Survey, and disability days questions at baseline and after 8 weeks. A randomly selected sub-sample of 30 subjects returned at 2 weeks to assess test retest reliability. Clinicians rated severity of symptoms, and both clinicians and patients rated change in gastroparesis-related symptoms over the 8 week study. RESULTS: The GCSI is based on three subscales: post-prandial fullness/early satiety (4 items); nausea/vomiting (3 items), and bloating (2 items). Internal consistency reliability was 0.84 for the GCSI total score and ranged from 0.83 to 0.85 for the subscale scores. Two week test retest reliability was 0.76 for the total score and ranged from 0.68 to 0.81 for subscale scores. Construct validity was supported, given that we observed significant relationships between clinician assessed symptom severity and GCSI total score, significant differences between gastroparesis and dyspepsia patients (n = 760) on GCSI total (p < 0.0001) and subscale scores (p < 0.03 to p < 0.0001), moderate and significant relationships between GCSI total and SF-36 scores, and significant associations between GCSI total score and reports of restricted activity and bed disability days. Patients with greater symptom severity, as rated by clinicians, reported more symptom severity on GCSI total score. GSCI total scores were responsive to changes in overall gastroparesis symptoms as assessed by clinicians (p < 0.0001) and patients (p = 0.0004). CONCLUSION: The findings of this study indicate that the GCSI is a reliable and valid instrument for measuring symptom severity in patients with gastroparesis.     

A co-designed consumer checklist to support people with eating disorders to locate evidence-based treatment

Identifying a therapist with expertise in treating eating disorders (EDs) can be a daunting process for a consumer, resulting in delayed access to evidence-based treatment, prolonging unnecessary suffering. We developed a consumer checklist designed to empower consumers to locate evidence-based treatment earlier. We co-designed (researchers and people with lived experience of an ED) a 15-item consumer checklist, for use in a meeting with a potential therapist. A survey about the checklist was sent out to people with lived experience and clinicians, seeking quantitative endorsement of each item's helpfulness in the checklist, and inviting qualitative feedback. Seventeen people with lived experience and 11 clinicians gave feedback. The items were rated as being helpful, with overlap between the two groups as to the most helpful items. Both groups rated the checklist as likely to help locate effective treatment earlier. The checklist forms a basis for a useful consumer tool in their treatment journey. Examination of its uptake and impact on outcomes for consumers seeking treatment is recommended.     

Development and Validation of the Nursing Home Care-related Quality of Life Scale

ObjectivesWe developed an instrument to assess nursing home residents' quality of life (QOL), with a focus on QOL-related factors modifiable through nursing home care, within the South Korean context; then, we tested its validity and reliability.DesignThe scale was constructed through a literature review, qualitative interviews, expert panel review, and a focus group interview; then, it was validated through survey research.Setting and participantsResidents (N = 667) from 42 nursing homes in South Korea participated in scale validation.MethodsNinety-nine items across 5 dimensions were initially drafted through a literature review and qualitative interview data. The expert panel review and a focus group interview yielded a scale with 31 items across 3 dimensions. Next, using survey data from nursing home residents, we examined the construct and concurrent validity and reliability (using Kuder-Richardson 20) of the 31-item QOL scale.ResultsThrough a series of factor analyses, the 31 items were reduced to 17 items across 2 dimensions: Environment and Services (13 items) and Social Interaction (4 items). This 17-item scale was further examined for model fitness and reliability. The scale had acceptable to good fit (root mean square error of approximation = 0.07 and 0.07; comparative fit index = 0.83 and 0.98 for Environment and Services and Social Interaction, respectively), and good concurrent validity and internal consistency (Kuder-Richardson 20 = 0.72).Conclusions and implicationsThis 17-item scale is a valid and reliable tool to assess nursing home residents' QOL in South Korea, with a focus on factors modifiable through nursing home care. Through more research to test its usefulness, validity, and reliability, the scale can be used as an index of nursing home care quality and contribute to the development of strategies to improve nursing home residents' QOL.     

Specialists Understanding of the Health Care Preferences of Chronically Ill Adolescents

PurposeDoctors must understand patients’ priorities to create an effective treatment partnership. Little is known about whether subspecialist pediatricians understand chronically ill adolescents’ preferences.MethodsA survey was conducted of 155 adolescents with chronic illnesses and 52 subspecialty physicians recruited from the same clinics of a children’s hospital. Adolescents and physicians rated the importance that adolescents place on items relating to quality of care and physician-patient communication styles using a previously validated measure.ResultsFor quality of care items, rank order correlation between physicians and patient responses was high (r = .63, p < .001) and both rated pain management items as most important. Physicians underestimated the importance adolescents placed on communicating with the physician as a friend and medical-technical aspects of care. For communication items, physicians’ responses were significantly different than adolescents for 13 of 17 items. Except for three items pertaining to autonomy, physician and patient responses were in the same direction, but adolescent responses were less extreme.ConclusionsPhysicians understood the importance of pain management to adolescents with chronic illnesses, but overestimated their desired level of autonomy. Asking adolescents for their preferences may be the first step in improving adolescents’ experience of care.     

Factors influencing student nurses' emotions toward children and their concept of motherhood

The purpose of this study is to investigate the factors influencing the emotions of student nurses toward children in order to apply the results to their practical training, since the opportunity for contact with children has been decreasing in recent years due to the declining birth rate. 278 student nurses completed a self-administered anonymous questionnaire that consisted of their background, such as living with children and experience of contact with children, their learning process, such as finished lectures and nursing training in child and maternal care or not, 28 items from Hanazawa's scale of emotions toward children, and 27 items for the concept of motherhood. The obtained data was analyzed using the SPSS 12.0J for Windows with the Spearman's correlation coefficient, Mann-Whitney U test and Kruskal Wallis test. The valid recovery was 121 (43.5%). The negative point for the concept of motherhood was significantly lower in the group that had finished nursing training in child and maternal care than in the group without such training. On the other hand, living with children and experience of contact with children did not influence their emotions toward children or concept of motherhood. The results of this study suggest that the experiences that student nurses have some deliberate and active contacts with children and mothers during their nursing training in child and maternal care have an influence on their concept of motherhood.     

Construction and validation of the Outpatient Health Care Usability Profile (OHCUP)

BackgroundThe Americans with Disabilities Act (ADA) requires health care facilities to provide equal access to patents with disabilities. Yet, people with disabilities experience many access barriers.Objective/hypothesisTo develop a valid, reliable, and user-friendly tool that measures the physical and environmental features of outpatient health care facilities. Instead of addressing full compliance with the ADA, the tool measures the essential features of an outpatient health care facility.MethodsThe project included an online survey of people with disabilities (reported elsewhere), work groups of people with disabilities prioritizing ADA administrative guidelines (ADAAG), ADA expert review, measure development, pilot testing and re-testing. ADA experts' ratings were summarized as Content Validity Ratios (CVR). Retained ADAAG items were organized into a survey instrument and tested by raters at outpatient health clinics. Inter-rater reliability was assessed using Cohen's Kappa coefficient and Gwet's AC1 statistic. Refinement and re-testing of the instrument was conducted.ResultsThe work groups narrowed the ADAG items from over 400 to 154 items. CVR ratings from ADA subject-matter experts reduced the items to 129. Inter-rater reliability for the pilot version was 0.61 (Kappa) and 0.88 (AC1). After refinement, inter-rater reliability was 0.77 (Kappa) and 0.90 (AC1). The items with the lowest reliability scores were re-examined, revised, and re-tested. Inter-rater reliability for the final version of the OHCUP was 0.89 (Kappa) and 0.97 (AC1).ConclusionsThe OHCUP is a valid and reliable tool for measuring the usability of health care facilities.     

Development of a scale for quality of care management process. A Delphi survey and studies on reliability and validity]

The purpose of this study was to develop a measurement tool for assessing quality in care management and to test its reliability and validity. A Delphi survey was initially administered on 96 experienced community health nurses, to improve the content validity of a questionnaire that was developed after three repeated rounds of data collection and content analysis. A total of 353 community health nurses, from 121 cities and towns across Japan, completed the mailed questionnaire. Cronbach's alpha value was more than 0.8, respectively, for all items in questionnaires, and for each factor, indicating internal consistency in reliability. Five factors were identified through factor analysis using a principal factor method with varimax rotation. These factors were good reflections of components classified by some researchers, indicating construct validity. In addition, care managers were grouped according to such criteria as their age and work experience. The QCM-P (Quality of Care Management-Process Measurement) score of each group was compared, as theoretically differences are expected. The questionnaire's validity was evidenced by significant differences in the QCM-P score among each group. Further studies on criterion-related validity and stability which relates to reliability are required. Thus, although further work is needed, QCM-P was found to have both reliability and validity at a permissible level as a scale for measuring the quality of care management process.     

Improving capacity to care for patients with intellectual and developmental disabilities: The value of an experiential learning model for family medicine residents

BackgroundPatients with intellectual and developmental disabilities (IDD) are more likely to experience poor health outcomes and family physicians receive inadequate training to provide appropriate care to this patient group. Little prior research has studied how to effectively train family physicians to care for patients with IDD.ObjectiveThe aim of this pilot study was to assess the value of adding an experiential component to didactic education strategies to improve family medicine resident perceived comfort, skills and knowledge related to caring for patients with IDD.MethodsStructured education programs for residents were implemented at three primary care practices in Ontario, Canada. Two practices received didactic information only (didactic-only group); one received didactic information and an experiential training model including clinical interactions and a written reflection on that experience (didactic plus experiential group). In this separate-sample pre-post design, residents were invited to complete a brief anonymous survey prior to and following the training assessing their perceived comfort, skills and knowledge related to patients with IDD.ResultsAt baseline, there were no significant differences between the two groups of residents. At follow up, the experiential group reported significantly higher levels of comfort, skills, and knowledge compared to baseline for most items assessed, while in the didactic-only group most items showed little or no improvement.ConclusionThis pilot study suggests that providing residents the opportunity to participate in clinical encounters with patients with IDD, as well as a structured process to reflect on such encounters, results in greater benefit than didactic training alone.     

Importance ratings on patient-reported outcome items for survivorship care: comparison between pediatric cancer survivors,parents, and clinicians

Purpose

To compare importance ratings of patient-reported outcomes (PROs) items from the viewpoints of childhood cancer survivors, parents, and clinicians for further developing short-forms to use in survivorship care.

Methods

101 cancer survivors, 101 their parents, and 36 clinicians were recruited from St. Jude Children’s Research Hospital. Participants were asked to select eight items that they deemed useful for clinical decision making from each of the four Patient-Reported Outcomes Measurement Information System Pediatric item banks. These item banks were pain interference (20 items), fatigue (23 items), psychological stress (19 items), and positive affect (37 items).

Results

Compared to survivors, clinicians rated more items across four domains that were statistically different than did parents (23 vs. 13 items). Clinicians rated five items in pain interference domain (ORs 2.33–6.01; p’s?<?0.05) and three items in fatigue domain (ORs 2.22–3.80; p’s?<?.05) as more important but rated three items in psychological stress domain (ORs 0.14–0.42; p’s?<?.05) and six items in positive affect domain (ORs 0.17–0.35; p’s?<?.05) as less important than did survivors. In contrast, parents rated seven items in positive affect domain (ORs 0.25–0.47; p’s?<?.05) as less important than did survivors.

Conclusions

Survivors, parents, and clinicians viewed importance of PRO items for survivorship care differently. These perspectives should be used to assist the development of PROs tools.

     

Context-bound communication skills training: development of a new method

OBJECTIVES: To examine how communication skills training might be integrated into everyday clinical practice in a manner that is acceptable to clinicians. DESIGN: General practitioners from 3 group practices agreed to take part, in turn, in a study of how to manage difficult consultations about antibiotic prescribing for acute respiratory infections. This provided the opportunity to conduct communication skills training in which lessons learned from one practice were taken into the next. SETTING: United Kingdom general practices. SUBJECTS: Three groups of general practitioners. FINDINGS: Difficulties with the acceptability of a traditional off-site workshop approach, using role play as the main teaching method, led to the development of a new training method (context-bound training), which proved to be practical and acceptable to experienced clinicians. The main features of the method were the delivery of training in the clinicians' place of work, and the transformation of their reported difficult cases into scenarios which they then encountered with a standardized simulated patient before and after brief seminars. Everyday clinical experience was kept in the foreground and 'communication skills' in the background. CONCLUSIONS: The method is acceptable to clinicians and adaptable to a range of clinical situations. It offers potential for improving the communication skills of clinicians both in hospital and primary care settings.     

Appropriate outcome measures for evaluating change in activities of daily living of elderly residents

PURPOSE: To identify functional dimensions related to changes in activities of daily living (ADL) and appropriate outcome measures for evaluating change in ADL of residents in a health care home for the elderly. SUBJECTS AND METHODS: The subjects were 123 residents living for three months or more in a health care home for the elderly in Osaka Prefecture. ADL on admission and three months these after were evaluated using four standardized outcome measures: ADL criteria for disabled elderly people developed by the Ministry of Health and Welfare (ADL criteria), the Barthel Index (BI), the Functional Independence Measure (FIM), and the Office of Population Censuses and Surveys (OPCS) scale. ResponsivenesS was also determined with the same four measures. The subjects were divided into two groups according to the frequency of functional training: twice or more and once or less per week. Appropriateness of the outcome measures was determined by examining score distributions and floor and ceiling effects, and by correlating changes in scores with frequency of functional training. RESULTS: The floor effect of the FIM was the smallest among the four outcome measures. The average FIM score on admission was significantly higher than that three months these after (70.7 +/- 30.8 and 71.6 +/- 30.6, respectively P < .05). In terms of funcTional dimensions of the BI, FIM and OPCS measures, the average FIM scores for transfer, walking, and stairs on admission were significantly higher than three months after admission (4.6 +/- 1.8 and 4.8 +/- 1.7 for transfer, 3.8 +/- 2.3 and 4.0 +/- 2.2 for walking, 3.0 +/- 2.1 and 3.2 +/- 2.1 for stairs, respectively, P < .05). Total FIM scores for the same three dimensions were the most responsive among the four outcome measures. Furthermore, when improvement in ADL was evaluated in terms of changes in total scores for the three FIM functional dimensions, those who received functional training twice or more per week showed a significantly higher rate of improvement in ADL than those who received functional training less frequently. CONCLUSION: FIM was found to be the most appropriate ADL measurement for evaluating changes in ADL of elderly residents. Regarding functional dimensions, locomotion in terms of transfer, walking and stairs was the most responsive. These findings indicate that the FIM functional dimension of locomotion is the most suitable for evaluating changes in ADL of elderly residents.     

Die psychometrischen Eigenschaften des Euro-Qol bei Rehabilitanden mit Erkrankungen des Stütz- und Bewegungsapparates

Purpose and study population: Health related quality of life (HRQL) is increasingly important for clinical and economic evaluation in rehabilitation care. This study investigates psychometric characteristics of the EuroQol (EQ-5D) questionnaire in patients with diseases of the musculoskeletal system. Study patients came from a German rehabilitation clinic and completed the EuroQol at admission and at discharge (n=149). A subgroup of 50 patients completed the quality assurance instrument of the pension fund of Württemberg and the Functional Independence Measure (FIM) together with the EuroQol. Another subgroup of 110 patients valued given health states for the examination of reliability. Methods: Floor and ceiling effects were used as indicators for discriminatory power. Construct validity was analysed by testing established hypotheses from the literature. Correlation with comparable items from the FIM and the instrument of the pension fund of Württemberg was calculated for criterion validity. For responsiveness, the change of items between admission and discharge was analysed for the three instruments. Intraclass correlation coefficients were used for test-retest reliability. Understanding problems in the valuation of given health states were examined in a feasibility test. Because of problems encountered with the original version of the EuroQol, the presentation of given health states was revised. Results: Neither floor effects nor significant ceiling effects could be observed, but there is a moderate medium level bias. Most of the hypotheses from the literature could be sustained. Criterion validity could be shown for central dimensions of HRQL, especially at discharge. The EuroQol is less responsive than the two other instruments however. With respect to feasibility and reliability, the preliminary results of the revised EuroQol version are very promising. Conclusion: In rehabilitation patients of the indication in question, the EuroQol seems to be a feasible instrument with acceptable psychometric properties and seems suitable for health state evaluation. It can add on to the existing disease specific or more sensitive generic instruments and may yield important information for economic evaluation studies.