The relationship between adjusted hospital mortality and the results of peer review.

This study assessed the relationship between the Health Care Financing Administration adjusted mortality rate for a hospital and the errors in care found by the peer review process. The three data sets used were: (1) the 1987-1988 completed reviews from 38 peer review organizations (PROs) of 4,132 hospitals and 2,035,128 patients; (2) all 1987 hospital mortality rates for Medicare patients as adjusted by HCFA for patient mix; and (3) the 1986 American Hospital Association Survey. The PRO data were used to compute the percentage of cases reviewed from each hospital confirmed by a reviewing physician to have a quality problem. The average percentage of confirmed problems was 3.73 percent with state rates ranging from 0.03 percent to 38.5 percent. The average within-state correlation between the problem rate and the adjusted mortality rate for all PROs was .19 (p < .0001), but the correlations were much higher for relatively homogeneous groups of hospitals, .42 for public hospitals and .36 for hospitals in large metropolitan statistical areas (MSAs). These results suggest that the HCFA adjusted hospital mortality rate and the PRO-confirmed problem rate are related methods to compare hospitals on the basis of quality of care. Both methods may compare quality better if used within a group of homogenous hospitals.     

Providers question PROs' effectiveness. Critics contend peer review organizations are too costly and fail to improve the quality of care

The Health Care Financing Administration (HCFA) established physician review organizations (PROs) to ensure that Medicare recipients receive care that is medically necessary, of high quality, and provided in the appropriate setting. While arguing that oversight is necessary, many healthcare professionals believe PROs do not accomplish what they were set up to do because physicians focus on the possibility of being penalized rather than on improving patient care. PRO critics claim that the program's peer reviewers are not peers of the physician under review and that, to be effective, they should come from the same local area. They contend the best peer review is conducted within the hospital. They believe intrafacility review can be more effective at bringing about improvement because hospital peer reviewers act as supportive, nonthreatening consultants. The confidentiality of the physician-patient relationship is another issue PRO critics raise. HCFA staffers say hospitalized Medicare patients are required to sign a waiver allowing inspection of their charts, but critics counter that waivers are only for the release of records for payment claims. Changes encouraging cooperation between PROs and hospitals could improve the PRO program and enhance quality of care.     

Toward a 21st century quality-measurement system for managed-care organizations

As the Nation's largest managed-care purchaser, the Health Care Financing Administration (HCFA) is working to develop a uniform data and performance-measurement system for all enrollees in managed-care plans. This effort will ultimately hold managed-care plans accountable for continuous improvement in the quality of care they provide and will provide information to consumers and purchasers to make responsible managed-care choices. The effort entails overhauling peer review organization (PRO) conduct of health maintenance organization (HMO) quality review, pilot testing a new HMO performance-measurement system, establishing criteria for Medicaid HMO quality-assurance (QA) programs, adapting employers' HMO performance reporting systems to the needs of Medicare and Medicaid, and participation in a new alliance between public and private sector managed-care purchasers to promote quality improvement and accountability for health plans.     

Quality improvement in health care. A brief history of the Medicare Peer Review Organization (PRO) initiative.

The Medicare Peer Review Organization (PRO) program began in the mid 1980s in response to concerns with medical necessity and quality of care of services delivered to the elderly and disabled, and paid for by the federal Medicare program. As part of their legislated oversight, PROs reviewed a random sample of hospital medical records. Using locally developed and maintained clinical criteria, PRO nurse and physician reviewers made determinations about the medical necessity of the inpatient stay and services, and identified issues with the quality of care delivered. Within 10 years of its initiation, however, criticisms of the PRO program, based in the reliability and validity of review findings, combined with national interest in quality improvement, led HCFA to refocus the program. PROs currently emphasize clinical and process quality improvement, through collaborative working relationships with providers and consumers.     

PRO update. Transition from the third to fourth scope of work.

The Health Care Financing Administration (HCFA) began implementation of the transition from the Third Peer Review Organization (PRO) Scope of Work to the Fourth PRO Scope of Work on October 1, 1991. PROs in 11 states are currently implementing newly-established medical review requirements under the "Scope of Work," the contract by which PROs are obligated to carry out their statutorily-mandated duties of determining the medical necessity, appropriateness and quality of care delivered to Medicare beneficiaries. Michigan's PRO (MPRO) is scheduled to implement requirements outlined in the Fourth Scope of Work beginning April 1, 1992. This article discusses the American Medical Association's (AMA) relationship to the PRO program and the transition from the Third to the Fourth Scope of Work including the status of the Uniform Clinical Data Set.     

Physician-reviewers' perceptions and judgments about quality of care.

OBJECTIVE: Although Peer Review Organizations (PROs) and researchers rely on physicians to assess quality of care, little is known about what physicians think about when they judge quality. We sought to identify features of individual cases that are associated with physicians' judgments. DESIGN: Using 1994 Medicare data, we selected hospitalizations for 1134 beneficiaries in 42 acute care hospitals in California and Connecticut. The sample was enriched with 17 surgical and six medical complications identified using diagnosis and procedure codes. PRO physicians confirmed quality problems using a structured implicit chart review instrument and provided written open-ended comments about each case. We coded physicians' comments for factors presumed to influence judgments about quality. RESULTS: In crude and adjusted comparisons, reviewers questioned quality more frequently in cases with serious or fatal outcomes, technical mishaps and inadequate documentation. Among surgical (but not medical) patients, they were less likely to record poor quality among patients presenting with an acute illness. CONCLUSION: Factors other than the adequacy of key processes of care are associated with physician-reviewers' judgments about quality.     

Case mix adjusted nursing-home reimbursement: a critical review of the evidence.

Nursing-home case mix adjusted payment systems typically base payments on estimates of patients' care needs, but to date the data on their effectiveness are ambiguous. Studies mainly show that access for patients most in need of care appears to improve under these systems. Case mix based payment systems have both positive and negative effects on quality of care and require compensating mechanisms for the potentially harmful incentives they can generate. On the positive side, nursing homes are paid more equitably; the negative aspect is reflected in higher costs, particularly for administration. A Health Care Financing Administration (HCFA) demonstration project may provide insights, but its limited number of predominantly small, rural, participating states, its tandem quality assurance system, and potentially confounding market variables may restrict the value of this project. We do not yet have the data to assess the impact of instituting case mix adjustment systems.     

The role of the principal clinical coordinator in the Health Care Financing Administration's Health Care Quality Improvement Initiative

The Health Care Quality Improvement Initiative (HCQII) is the Health Care Financing Administration's latest approach to quality management by peer review organizations (PROs) of the health care delivered to Medicare beneficiaries. The principal clinical coordinator (PCC) of each PRO, a physician with both clinical and methodological expertise, has the overall responsibility for all HCQII-related activities. The PCC works with a clinical coordinating team to analyze patterns of care and provide feedback about these patterns to the medical community for the purpose of improving the quality of care. Each PRO provides its PCC with the necessary infrastructure and expertise to conduct pattern analyses and implement the dissemination process. However, the effectiveness of the HCQII will depend largely on the PCC's ability to maintain the full support and cooperation of the local medical community. The success of the PCC role under the HCQII may enable it to serve as a useful model of physician leadership in the quality oversight organizations that will accompany national health care reform.     

Methods for Developing Patient-Reported Outcome-Based Performance Measures (PRO-PMs)

ObjectiveTo recommend methods for assessing quality of care via patient-reported outcome-based performance measures (PRO-PMs) of symptoms, functional status, and quality of life.MethodsA Technical Expert Panel was assembled by the American Medical Association–convened Physician Consortium for Performance Improvement. An environmental scan and structured literature review were conducted to identify quality programs that integrate PRO-PMs. Key methodological considerations in the design, implementation, and analysis of these PRO-PM data were systematically identified. Recommended methods for addressing each identified consideration were developed on the basis of published patient-reported outcome (PRO) standards and refined through public comment. Literature review focused on programs using PROs to assess performance and on PRO guidance documents.ResultsThirteen PRO programs and 10 guidance documents were identified. Nine best practices were developed, including the following: provide a rationale for measuring the outcome and for using a PRO-PM; describe the context of use; select a measure that is meaningful to patients with adequate psychometric properties; provide evidence of the measure’s sensitivity to differences in care; address missing data and risk adjustment; and provide a framework for implementation, interpretation, dissemination, and continuous refinement.ConclusionMethods for integrating PROs into performance measurement are available.     

Patient-reported outcomes measurement and management with innovative methodologies and technologies

Successful integration of modern psychometrics and advanced informatics in patient-reported outcomes (PRO) measurement and management can potentially maximize the value of health outcomes research and optimize the delivery of quality patient care. Unlike the traditional labor-intensive paper-and-pencil data collection method, item response theory-based computerized adaptive testing methodologies coupled with novel technologies provide an integrated environment to collect, analyze and present ready-to-use PRO data for informed and shared decision-making. This article describes the needs, challenges and solutions for accurate, efficient and cost-effective PRO data acquisition and dissemination means in order to provide critical and timely PRO information necessary to actively support and enhance routine patient care in busy clinical settings.     

Assessment Methods for Patient-Reported Outcomes

The methods used to collect information on patient-reported outcomes (PROs) can affect the validity, reliability and sensitivity of the data. This review examines the influence of a variety of methodological issues that occur. In particular, when patients are asked to rely on their memory to aggregate and summarize their experience, a variety of inaccuracies and biases can affect the data. As the extent and magnitude of recall bias have become more well-known, researchers have increasingly turned toward collecting data in the field, closer in time to events and experiences of interest. Diary methods require patients to self-monitor their experiences, disease episodes and healthcare utilization over time. The collection of diary data from patients places a premium on adherence to the data collection protocol. Recent evidence suggests that patients routinely fail to adhere to diary protocols, thus introducing retrospective error and biases into the data. Reasons for patients’ non-adherence to data collection protocols include their desire to appear to be ‘good’ patients which can result in back-filling PRO entries before a site visit, simply forgetting, and data collection methods that fail to help patients be adherent to the data collection protocol. Principles that can be used to enhance patient adherence to PRO data collection methods include good patient training, creating simple and intuitive user interfaces for PRO assessments, and actively driving the protocol by helping patients remember to complete PRO measures as required by the protocol.     

Integrating the Use of Patient‐Reported Outcomes for Both Clinical Practice and Performance Measurement: Views of Experts from 3 Countries

Context

Patient-reported outcomes (PROs) can play an important role in patient-centered health care by focusing on the patient''s health goals guiding therapeutic decisions. When aggregated, PROs also can be used for other purposes, including comparative effectiveness research, practice improvement, assessment of the performance of clinicians and organizations, and as a metric for value-based payments. The feasibility of integrating the use of PROs for these various purposes on a wide scale has not yet been demonstrated. Our study was conducted to inform policymakers of prudent next steps for implementing PROs in clinical practice and performance measurement programs in order to maximize their impact on the quality of health care.

Methods

We conducted a qualitative study, interviewing 58 experts and leaders from 37 organizations (response rate: 88%) in the United States, England, and the Netherlands. Respondents included clinical practitioners (n = 30), measure developers (n = 11), and leaders of performance measurement programs (n = 17). We used a qualitative content analysis to assess current strategies for applying PROs in clinical practice and performance measurement and to identify barriers to and facilitators of further implementation.

Findings

The use of PROs in clinical practice and for performance measurement has developed both separately and in parallel. Experts across the stakeholder spectrum support the collection of PRO data in an integrated manner that would enable using the data for these distinct purposes. We identified 2 main concerns about the feasibility for integrated use of PRO data: the complexity of establishing routine data collection and the tension among stakeholders when using PRO data for different purposes. These contrasting stakeholder views suggested varying interests among clinicians, measure developers, and purchasers of care.

Conclusions

Data collection approaches that support the use of PROs in health care are underdeveloped, need better integration with clinical care, and must be tailored to the characteristics of the health care system. Enabling the sustainable use of PROs will require a shared vision of clinical professionals, purchasers, and patients, with a prudent selection of the steps in implementing PROs that will maximize their impact on the quality of health care.     

HIM resources--practice tools: mortality review

As discussed here, HCFA mortality information is not a simple index of hospital performance. The data included are just one indicator of patient outcome for just one group of the total in-patient hospital population. With growing public interest in the quality of healthcare, however, it is in a hospital's best interest to analyze HCFA mortality data and respond to public inquiries regarding quality of care.     

A methodology for targeting hospital cases for quality of care record reviews.

We tested the efficacy of selected case characteristics in targeting quality of care problems for medical record review. The case characteristics, all of which apply to patients who die in a hospital, consist primarily of procedures and DRGs (diagnosis-related groups) for which death rarely occurs, and a set of complications of surgical care. All characteristics are obtainable from combinations of the principal and secondary diagnoses and procedures in the case, and are available from discharge abstracts. The presence of a quality of care problem is confirmed through a review of the medical record by a nurse and two or more physicians. A logistic regression model that controls for various patient and hospital variables is used as a measure of each of the proposed case characteristics. The results indicate that most of the characteristics are associated with higher percentages of quality of care problems than cases chosen at random, and that the methodology has promise as a tool for targeting cases for medical record review.     

Medicaid managed care encounter data: what, why, and where next?

Managed care now serves 23 percent of the Medicaid population. With the shift to capitation, the fee-for-service (FFS) billing mechanism that has generated much of the administrative data used in policy planning and research no longer exists. This article provides an overview of the types of encounter data currently being required for plans and the problems and issues with providing and analyzing such data. It is based on a review of documentation and interviews with representatives of nine States and the Health Care Financing Administration (HCFA). The article concludes by providing recommendations for HCFA, States, and plans in creating and improving encounter data systems.     

Validation of Electronic Systems to Collect Patient-Reported Outcome (PRO) Data—Recommendations for Clinical Trial Teams: Report of the ISPOR ePRO Systems Validation Good Research Practices Task Force

Outcomes research literature has many examples of high-quality, reliable patient-reported outcome (PRO) data entered directly by electronic means, ePRO, compared to data entered from original results on paper. Clinical trial managers are increasingly using ePRO data collection for PRO-based end points. Regulatory review dictates the rules to follow with ePRO data collection for medical label claims. A critical component for regulatory compliance is evidence of the validation of these electronic data collection systems. Validation of electronic systems is a process versus a focused activity that finishes at a single point in time. Eight steps need to be described and undertaken to qualify the validation of the data collection software in its target environment: requirements definition, design, coding, testing, tracing, user acceptance testing, installation and configuration, and decommissioning. These elements are consistent with recent regulatory guidance for systems validation. This report was written to explain how the validation process works for sponsors, trial teams, and other users of electronic data collection devices responsible for verifying the quality of the data entered into relational databases from such devices. It is a guide on the requirements and documentation needed from a data collection systems provider to demonstrate systems validation. It is a practical source of information for study teams to ensure that ePRO providers are using system validation and implementation processes that will ensure the systems and services: operate reliably when in practical use; produce accurate and complete data and data files; support management control and comply with any existing regulations. Furthermore, this short report will increase user understanding of the requirements for a technology review leading to more informed and balanced recommendations or decisions on electronic data collection methods.     

Issues in the design of Internet-based systems for collecting patient-reported outcomes

Background and objectives Although there is a growing interest in using patient-reported outcomes (PRO) to monitor disease progression and/or therapeutic response, to improve care, and to screen for physical or psychosocial problems in routine clinical practice, PRO instruments can be difficult to administer, score, and interpret in this setting. Internet-based approaches to PRO collection may help overcome these obstacles. This paper discusses the rationale for using the Internet for routine PRO collection, summarizes relevant literature and ongoing projects, and raises several key design and development issues that should guide further efforts in this area. Major findings A small number of Internet-based PRO collection applications have been or are currently being developed. The major characteristics of several of these projects are reviewed and summarized. Successful Internet-based PRO collection applications must address patient and clinician-specific needs related to workflow and to the way in which results are presented. A growing number of instruments have been adapted for and evaluated in a web-based format. Conclusions Collecting PROs via the Internet has the potential to overcome many of the challenges associated with efforts to routinely use PROs in the clinical encounter. James B. Jones, Claire F. Snyder, and Albert W. Wu for the “Website for Outpatient QOL Assessment” Research Network.     

A survey of newspaper coverage of HCFA hospital mortality data

A study that assessed newspaper coverage of the 1986 Hospital Mortality Data for Medicare Patients released by the Health Care Financing Administration (HCFA) of the U.S. Department of Health and Human Services is described. Media interpretation of Federal information about the quality of hospital medical care is also discussed. A sample of 68 articles from newspapers serving urban areas of various sizes in all regions of the United States was analyzed. Articles were coded into classifications according to how the news was played, headline bias (positive-negative-neutral), hospital mentions, quote sources, explanations for excessively high mortality rates, urban area population, and geographic region. The findings indicated that HCFA's release of the 1986 hospital mortality data received heavy news coverage. There were twice as many negative headlines as positive ones, although nearly 95 percent of the hospitals had mortality rates within expected ranges. Quotes from representatives of hospitals predominated in the newspaper articles, and they often blamed some aspect of the HCFA data for higher-than-expected mortality rates. Newspaper attention to the quality of hospital care clearly raised consumer awareness of the idea that health care quality can vary. The newspaper articles, however, provided no guidance on obtaining valid data or on using it to make health care choices.