The Outcome and Assessment Information Set (OASIS): A Review of Validity and Reliability

The Outcome and Assessment Information Set (OASIS) is the patient-specific, standardized assessment used in Medicare home health care to plan care, determine reimbursement, and measure quality. Since its inception in 1999, there has been debate over the reliability and validity of the OASIS as a research tool and outcome measure. A systematic literature review of English-language articles identified 12 studies published in the last 10 years examining the validity and reliability of the OASIS. Empirical findings indicate the validity and reliability of the OASIS range from low to moderate but vary depending on the item studied. Limitations in the existing research include: nonrepresentative samples; inconsistencies in methods used, items tested, measurement, and statistical procedures; and the changes to the OASIS itself over time. The inconsistencies suggest that these results are tentative at best; additional research is needed to confirm the value of the OASIS for measuring patient outcomes, research, and quality improvement.     

妇幼保健机构综合绩效评价指标体系构建

目的 探索构建妇幼保健机构综合绩效评价指标体系,为妇幼保健机构绩效管理提供思路和依据.方法 采用文献回顾、专家论证法、Delphi专家咨询法、离散趋势法、相关分析法、主成分分析与因子分析法和聚类分析法等建立指标体系,采用专家评分法和CRITIC法主客观权衡计算指标权重系数,并对指标体系的一致性信度及内部结构效度进行检验.结果 根据绩效评价指标体系框架,设定医疗保健工作质量、医疗保健工作效率、公共卫生服务质量和效率、社会效益、经济效益、发展潜力6大类一级指标.结论 通过科学研究,可以建立一套科学合理的适用于我国不同级别妇幼保健机构的综合绩效评价指标体系.     

The development and validation of the primary care satisfaction survey for women.

Assessing patient satisfaction with health care is becoming an integral component of quality monitoring in health care systems, but existing tools typically were developed to minimize differences related to gender. This paper reports the development and psychometric properties of a new survey instrument to measure women's satisfaction with their primary care. A multisite, cross-sectional validation survey of 1,202 women receiving care in primary care settings in Michigan, North Carolina, and Pennsylvania was conducted. Item response theory (IRT) and factor analysis methods were used to identify three scales in the Primary Care Satisfaction Survey for Women (PCSSW): Communication, Administration and Office Procedures, and Care Coordination and Comprehensiveness. Internal consistency reliability is reported, as well as convergent validity in relation to two generic measures (Medical Outcomes Study [MOS] Visit Satisfaction and Consumer Assessment of Health Plans Survey [CAHPS] overall quality rating); discriminant validity in relation to groups expected to have differing satisfaction levels based on previous literature; predictive validity in relation to behavioral intentions; and explanatory power in overall satisfaction ratings. The 24-item PCSSW may be self-administered or conducted by telephone and may be used in studies to evaluate or improve the quality of primary care for women.     

Development of a New Questionnaire to Assess Patient Perceptions of Cancer-Related Fatigue: Item Generation and Item Reduction

Objectives:  Existing instruments that measure the impact of cancer-related fatigue on health-related quality of life do not usually incorporate the attitudes, beliefs and perspectives of patients. This study aimed to develop an instrument to measure the impact of cancer-related fatigue on the health-related quality of life of cancer patients.
Methods:  Items were generated from a literature review, focus groups of cancer patients and meetings with oncologists. Potential items were administered to cancer patients to facilitate item reduction, which was based on clinimetric and psychometric analyses and qualitative criteria. A preliminary assessment of feasibility, reliability and validity of the retained items was performed.
Results:  An initial pool of 75 items was administered to 238 cancer patients. Fifty items were eliminated after statistical analysis and 13 in response to expert opinion, resulting in a provisional instrument with 12 items in 3 dimensions. These displayed acceptable internal consistency (Cronbach's alpha, 0.78–0.92) and their overall score was associated with fatigue intensity, extent of disease, intention of treatment and need of caregivers.
Conclusion:  The newly developed questionnaire, which measures the impact of cancer-related fatigue on oncology patients, has shown satisfactory feasibility, reliability and validity.     

Quality of life measurement in paediatric and adolescent populations with HIV: a review of the literature

Purpose   To review the quality and utility of currently available self-report generic quality of life (QOL) and health-related quality of life (HRQOL) measures for use with children and adolescents with human immunodeficiency virus and/or acquired immunodeficiency syndrome (HIV/AIDS).
Methods   Literature searches were conducted to identify QOL and HRQOL measures developed for, adapted for, or otherwise used with paediatric and adolescent populations with HIV/AIDS. The quality of measures (i.e. item generation techniques, instrument properties including reliability, validity and responsiveness) were compared and critically evaluated.
Results   Nineteen QOL/HRQOL measures were identified. Item content was generated from the respondent (adult) population in only eight (42%) measures. Seventeen (90%) measures reported internal reliability in the accepted range between 0.70 and 0.90 and four (21%) reported reproducibility statistics in this range. Although validity was reported for 19 (100%) measures, only six (32%) showed evidence for three or more properties, with construct validity being the most commonly reported aspect. The authors of eight (42%) measures reported evidence for responsiveness.
Conclusions   While almost all measures reviewed demonstrated adequate psychometric properties, only one-third demonstrated all aspects of validity, and less than half demonstrated responsiveness. None included paediatric or adolescent populations with HIV/AIDS in their development, neglecting to obtain input from target respondents in item generation to determine what health-related and daily functioning factors are of importance to them. Despite noted limitations, the AUQUEI or the SWED-QUAL appear the best currently available generic measure, and the MQOL-HIV the preferred disease-specific measure, at least for use with older adolescents/young adults.     

National Release of the Nursing Home Quality Report Cards: Implications of Statistical Methodology for Risk Adjustment

Objective. To determine how alternative statistical risk-adjustment methods may affect the quality measures (QMs) in nursing home (NH) report cards.
Data Sources/Study Settings. Secondary data from the national Minimum Data Set files of 2004 and 2005 that include 605,433 long-term residents in 9,336 facilities.
Study Design. We estimated risk-adjusted QMs of decline in activities of daily living (ADL) functioning using classical, fixed-effects, and random-effects logistic models. Risk-adjusted QMs were compared with each other, and with the published QM (unadjusted) in identifying high- and low-quality facilities by either the rankings or 95 percent confidence intervals of QMs.
Principal Findings. Risk-adjusted QMs showed better overall agreement (or convergent validity) with each other than did the unadjusted versus each adjusted QM; the disagreement rate between unadjusted and adjusted QM can be as high as 48 percent. The risk-adjusted QM derived from the random-effects shrinkage estimator deviated nonrandomly from other risk-adjusted estimates in identifying the best 10 percent facilities using rankings.
Conclusions. The extensively risk-adjusted QMs of ADL decline, even when estimated by alternative statistical methods, show higher convergent validity and provide more robust NH comparisons than the unadjusted QM. Outcome rankings based on ADL decline tend to show lower convergent validity when estimated by the shrinkage estimator rather than other statistical methods.     

A systematic review of reliability and objective criterion-related validity of physical activity questionnaires

ABSTRACT: Physical inactivity is one of the four leading risk factors for global mortality. Accurate measurement of physical activity (PA) and in particular by physical activity questionnaires (PAQs) remains a challenge. The aim of this paper is to provide an updated systematic review of the reliability and validity characteristics of existing and more recently developed PAQs and to quantitatively compare the performance between existing and newly developed PAQs.A literature search of electronic databases was performed for studies assessing reliability and validity data of PAQs using an objective criterion measurement of PA between January 1997 and December 2011. Articles meeting the inclusion criteria were screened and data were extracted to provide a systematic overview of measurement properties. Due to differences in reported outcomes and criterion methods a quantitative meta-analysis was not possible.In total, 31 studies testing 34 newly developed PAQs, and 65 studies examining 96 existing PAQs were included. Very few PAQs showed good results on both reliability and validity. Median reliability correlation coefficients were 0.62--0.71 for existing, and 0.74--0.76 for new PAQs. Median validity coefficients ranged from 0.30--0.39 for existing, and from 0.25--0.41 for new PAQs.Although the majority of PAQs appear to have acceptable reliability, the validity is moderate at best. Newly developed PAQs do not appear to perform substantially better than existing PAQs in terms of reliability and validity. Future PAQ studies should include measures of absolute validity and the error structure of the instrument.     

What is adequate health care and how can quality of care be improved?

Attempts to improve patient care, its increasing cost and the aggressive malpractice environment have highlighted the need for standards of professional accountability. However, current measures of quality of care have mostly been met with skepticism by the medical community. These measures have been criticized for their uncertain validity and for focusing on secondary aspects of service that measure what is minimally acceptable. The objective of this essay is to review quality improvement methods that have been reported to be feasible, effective and acceptable by practicing physicians. The successful implementation of these methods seems to be related to their being nonintrusive, nonthreatening, and based on agreed upon standards of care. We believe that these three features are essential for a continuous quality improvement process in health care.     

ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research

Purpose

An essential aspect of patient-centered outcomes research (PCOR) and comparative effectiveness research (CER) is the integration of patient perspectives and experiences with clinical data to evaluate interventions. Thus, PCOR and CER require capturing patient-reported outcome (PRO) data appropriately to inform research, healthcare delivery, and policy. This initiative’s goal was to identify minimum standards for the design and selection of a PRO measure for use in PCOR and CER.

Methods

We performed a literature review to find existing guidelines for the selection of PRO measures. We also conducted an online survey of the International Society for Quality of Life Research (ISOQOL) membership to solicit input on PRO standards. A standard was designated as “recommended” when >50 % respondents endorsed it as “required as a minimum standard.”

Results

The literature review identified 387 articles. Survey response rate was 120 of 506 ISOQOL members. The respondents had an average of 15 years experience in PRO research, and 89 % felt competent or very competent providing feedback. Final recommendations for PRO measure standards included: documentation of the conceptual and measurement model; evidence for reliability, validity (content validity, construct validity, responsiveness); interpretability of scores; quality translation, and acceptable patient and investigator burden.

Conclusion

The development of these minimum measurement standards is intended to promote the appropriate use of PRO measures to inform PCOR and CER, which in turn can improve the effectiveness and efficiency of healthcare delivery. A next step is to expand these minimum standards to identify best practices for selecting decision-relevant PRO measures.     

Methods Used to Streamline the CAHPS® Hospital Survey

Objective. To identify a parsimonious subset of reliable, valid, and consumer-salient items from 33 questions asking for patient reports about hospital care quality.
Data Source. CAHPS^® Hospital Survey pilot data were collected during the summer of 2003 using mail and telephone from 19,720 patients who had been treated in 132 hospitals in three states and discharged from November 2002 to January 2003.
Methods. Standard psychometric methods were used to assess the reliability (internal consistency reliability and hospital-level reliability) and construct validity (exploratory and confirmatory factor analyses, strength of relationship to overall rating of hospital) of the 33 report items. The best subset of items from among the 33 was selected based on their statistical properties in conjunction with the importance assigned to each item by participants in 14 focus groups.
Principal Findings. Confirmatory factor analysis (CFA) indicated that a subset of 16 questions proposed to measure seven aspects of hospital care (communication with nurses, communication with doctors, responsiveness to patient needs, physical environment, pain control, communication about medication, and discharge information) demonstrated excellent fit to the data. Scales in each of these areas had acceptable levels of reliability to discriminate among hospitals and internal consistency reliability estimates comparable with previously developed CAHPS instruments.
Conclusion. Although half the length of the original, the shorter CAHPS hospital survey demonstrates promising measurement properties, identifies variations in care among hospitals, and deals with aspects of the hospital stay that are important to patients' evaluations of care quality.