Impact of an acute myocardial infarction report card in Ontario, Canada.

OBJECTIVES: Acute myocardial infarction (AMI) 'report cards' are being developed using administrative databases in many jurisdictions, but little is known about their acceptance by and their usefulness to the medical community. The purpose of this study was to determine the impact of the publication of Cardiovascular Health and Services in Ontario: An ICES Atlas (Naylor CD, Slaughter P. (eds), 1999, Toronto: ICES), the first report featuring hospital-specific AMI performance measures to be published in Canada. DESIGN: We conducted a mail survey of physicians at Ontario hospitals to determine their views on the usefulness of various atlas performance measures for assessing and improving quality of care, the types of quality initiatives launched at their hospital in response to the atlas, and their views on the concept and limitations of reporting hospital-specific AMI mortality data. RESULTS: Respondents to the survey indicated that information on process of care measures such as post-infarction beta-blocker and angiotensin-converting enzyme (ACE) inhibitor use, and cardiac procedure waiting times were the most useful, and outcomes data (e.g. 30-day and 1-year risk-adjusted AMI mortality rates) the least useful of the multiple performance measures published in the atlas (P = 0.0385). Fifty-four percent of respondents reported launching one or more quality of care initiatives at their hospital in response to the atlas. The majority of respondents (65%) indicated that they support the public release of hospital-specific AMI mortality data, although many had concerns about potential miscoding in administrative databases and the adequacy of risk-adjustment methods. CONCLUSION: The publication of the first AMI report card in Canada stimulated quality of care initiatives at many Ontario hospitals. Inclusion of performance measures other than mortality in health care report cards may lead to greater acceptance and use by the medical community.     

Do markets respond to quality information? The case of fertility clinics

Although policymakers have increasingly turned to provider report cards as a tool to improve health care quality, existing studies provide mixed evidence on whether they influence consumer choices. We examine the effects of providing consumers with quality information in the context of fertility clinics providing Assisted Reproductive Therapies (ART). We report three main findings. First, clinics with higher birth rates had larger market shares after the adoption of report cards relative to before. Second, clinics with a disproportionate share of young, relatively easy-to-treat patients had lower market shares after adoption versus before. This suggests that consumers take into account information on patient mix when evaluating clinic outcomes. Third, report cards had larger effects on consumers and clinics from states with ART insurance coverage mandates. We conclude that consumers respond to quality report cards when choosing among providers of ART.     

Using report cards to grade health plan quality

Judging the quality of health care that a plan delivers so consumers can make informed purchasing decisions is at the heart of the major federal health reform proposals. Health care "report cards" are an important step in this direction. But they cannot, by themselves, inform consumers or improve health system performance.     

Health claims data as a strategy and tool in disease management

A comprehensive definition of disease management provides an opportunity to track a population of patients across the entire continuum of a condition, from wellness through disease and disability, so that improvements in health status and quality of life and efficiencies in the application of health care resources can be demonstrated. The need is great for information systems that can computerize clinical encounter, summarize, and apply the information to help identify opportunities for improvement in the performance of quality and cost control, monitor processes of care, and report outcomes that are meaningful to the organization. By tracking health care charges as a proxy for the application of health care resources, health claim data analyses can identify conditions for disease management, facilitate provider buy-in, develop the disease management program, monitor interventions, and report outcomes.     

HMOs: responding to bad report cards

Differentiating one health plan from another is becoming increasingly difficult. But a plethora of report cards on plans' performance are now available to help purchasers pick and choose managed care for their members. When a health plan receives a low score--or a score lower than its competitors, be it ever so slight--does it respond appropriately? Are leaders adequately following up to determine the significance of the scoring, rectify the problem--and communicate that to purchasers? Is quality improvement an outcome of publishing report cards? TQL spoke with health plan leaders, healthcare consultants and purchasers to find out what some plans are doing--and what many should be doing.     

Hospital report cards: Quality competition and patient selection

Hospital ‘report cards’ policies involve governments publishing information about hospital quality. Such policies often aim to improve hospital quality by stimulating competition between hospitals. Previous empirical literature lacks a comprehensive theoretical framework for analysing the effects of report cards. We model a report card policy in a market where two hospitals compete for patients on quality under regulated prices. The report card policy improves the accuracy of the quality signal observed by patients. Hospitals may improve their published quality scores by costly quality improvement or by selecting healthier patients to treat. We show that increasing information through report cards always increases quality and only sometimes induces selection. Report cards are more likely to increase patient welfare when quality scores are well risk-adjusted, where the cost of selecting patients is high, and the cost of increasing quality is low.     

Public reporting on quality in the United States and the United Kingdom

The public reporting of comparative information about health care quality is becoming an accepted way of improving accountability and quality. Quality report cards have been prominent in the United States for more than a decade and are a central feature of British health system reform. In this paper we examine the common challenges and differences in implementation of the policy in the two countries. We use this information to explore some key questions relating to the content, target audience, and use of published information. We end by making specific recommendations for maximizing the effectiveness of public reporting.     

Health care consumer reports: an evaluation of consumer perspectives

There has been a proliferation of health care consumer reports, also known as "consumer guides," "report cards," and "performance reports," which are designed to assist consumers in making more informed health care decisions. While there is evidence that providers use such reports to identify and make changes in practice, thus improving the quality of care, there is little empirical evidence on how consumer guides/report cards are used by consumers. This study fills that gap by surveying 925 patients as they wait for ambulatory care in several clinics in a midwestern city. Findings indicate that consumers are selective in their use of these reports and quickly identify those sections of the report of most interest to them. Report developers should take precautions to ensure such reports are viewed as credible sources of health care information.     

Patients' preferences for technical versus interpersonal quality when selecting a primary care physician

OBJECTIVE: To assess patients' use of and preferences for information about technical and interpersonal quality when using simulated, computerized health care report cards to select a primary care provider (PCP). DATA SOURCES/STUDY SETTING: Primary data collected from 304 adult consumers living in Los Angeles County in January and February 2003. STUDY DESIGN/DATA COLLECTION: We constructed computerized report cards for seven pairs of hypothetical individual PCPs (two internal validity check pairs included). Participants selected the physician that they preferred. A questionnaire collected demographic information and assessed participant attitudes towards different sources of report card information. The relationship between patient characteristics and number of times the participant selected the physician who excelled in technical quality are estimated using an ordered logit model. PRINCIPAL FINDINGS: Ninety percent of the sample selected the dominant physician for both validity checks, indicating a level of attention to task comparable with prior studies. When presented with pairs of physicians who varied in technical and interpersonal quality, two-thirds of the sample (95 percent CI: 62, 72 percent) chose the physician who was higher in technical quality at least three out of five times (one-sample binomial test of proportion). Age, gender, and ethnicity were not significant predictors of choosing the physician who was higher in technical quality. CONCLUSIONS: These participants showed a strong preference for physicians of high technical quality when forced to make tradeoffs, but a substantial proportion of the sample preferred physicians of high interpersonal quality. Individual physician report cards should contain ample information in both domains to be most useful to patients.     

Determinants of nursing home choice: Does reported quality matter?

Quality report cards addressing information asymmetry in the health care market have become a popular strategy used by policymakers to improve the quality of care for older people. Using individual level data from the largest German sickness fund merged with institutional level data, we examine the relationship between reported nursing home quality, as measured by recently introduced report cards, nursing home prices, nursing home's location, and the individual choice of nursing homes. Report cards were stepwise introduced as of 2009, and we use a sample of 2010 that includes both homes that had been evaluated at that time and that had not yet been. Thus, we can distinguish between institutions with above and below average ratings as well as nonrated nursing homes. We find that the probability of choosing a nursing home decreases in distance and price. However, we find no economically significant effect of reported quality on individuals' choice of nursing homes.     

Making the grade: update on report card initiatives for 1997

Report cards for healthcare services are increasingly in the news, offering the hope that objective information on the quality of health plan and providers services will eventually enable purchasers and consumers to make selections based on true value. Following is a series of five brief articles that review ongoing report card initiatives in private and public sectors of the behavioral healthcare system. The first four articles review actual report cards designed to hold organizations--particularly managed care--accountable for the quality of their services. The last article reviews research on performance measurement across all segments of the behavioral healthcare industry.     

Agencies start TalkingQuality,give hospitals tools to educate public

New government web site designed to help health systems and hospitals provide consumers with quality report cards and other information regarding health care quality measurement.     

Increasing the impact of health plan report cards by addressing consumers' concerns

Most plan report cards that compare the performance of health plans have framed the decision about plan choice as an opportunity to get better-quality care. This study uses a controlled experimental design to examine the effect of reframing the health plan choice decision to one that emphasizes protecting oneself from possible risk. The findings show that framing the health plan decision using a risk message has a consistent and significant positive impact on how consumers comprehend, value, and weight comparative performance information.     

Costs and effects of new professional roles: Evidence from a literature review

One way in which governments are seeking to improve the efficiency of the health care sector is by redesigning health services to contain labour costs. The aim of this study was to investigate the impact of new professional roles on a wide range of health service outcomes and costs.A systematic literature review was performed by searching in different databases for evaluation papers of new professional roles (published 1985–2013). The PRISMA checklist was used to conduct and report the systematic literature review and the EPHPP-Quality Assessment Tool to assess the quality of the studies.Forty-one studies of specialist nurses (SNs) and advanced nurse practitioners (ANPs) were selected for data extraction and analysis. The 25 SN studies evaluated most often quality of life (10 studies), clinical outcomes (8), and costs (8). Significant advantages were seen most frequently regarding health care utilization (in 3 of 3 studies), patient information (5 of 6), and patient satisfaction (4 of 6). The 16 ANP studies evaluated most often patient satisfaction (8), clinical outcomes (5), and costs (5). Significant advantages were seen most frequently regarding clinical outcomes (5 of 5), patient information (3 of 4), and patient satisfaction (5 of 8).Promoting new professional roles may help improve health care delivery and possibly contain costs. Exploring the optimal skill-mix deserves further attention from health care professionals, researchers and policy makers.     

Loss function-based evaluation of physician report cards

Report cards classifying physicians into performance tiers are central to health care quality improvement initiatives. Misclassification is a concern since physicians often have small patient panels for standard performance measures. Given that report cards are used for different purposes by different stakeholders, we specify loss functions and evaluate the potential cost of misclassification for physician report card designs. Monte Carlo simulation to explore misclassification risk and cost for four illustrative physician report card designs and three loss functions representing overall misclassification, patient, and pay-for-performance program perspectives. True physician performance is simulated under a beta-binomial model with parameters yielding simulated true scores resembling previously reported estimates. Misclassification risk varied across report card designs. Overall misclassification risk increased with the number of performance tiers for our simulated scenarios. However, the relationship between misclassification cost and number of tiers was inconsistent across the loss functions. The report card with the lowest misclassification cost varied across stakeholders. Within stakeholder, the costs of a two-tier report card with a high or low hurdle (25th and 75th percentile, respectively) varied. Loss functions and report card designs are illustrative and not intended to exhaustively catalog all possibilities. Little guidance exists on misclassification costs from the patient perspective. Misclassification cost depends on how performance information will be used and by whom. Selecting the lowest-cost design for a given stakeholder could maximize the usefulness of physician performance data. Misclassification cost could guide report card design, improving the usefulness of a report card for one stakeholder without disadvantaging others .     

Designing a system for managing the performance of mental health managed care: An example from new york state's prepaid mental health plan

The organization, financing, and delivery of publicly funded behavioral health services are undergoing massive changes nationwide. Managed care principles and practices are being implemented widely and are being relied on increasingly to meet the challenges of containing costs and improving service effectiveness. To meet these goals, comprehensive systems are under development for measuring and reporting outcomes experienced by individuals who receive services and for assessing the impact of managed care strategies on the service delivery system. This article presents an example from the Prepaid Mental Health Program in New York State. It highlights the development, implementation, and early experiences with the plan's performance management system for public sector managed behavioral health, a basis for continuous quality improvement activities and information reporting products such as report cards. Policy, administrative, and financial implications are illuminated.     

The impact of unmeasured clinical variables on the accuracy of hospital report cards: a Monte Carlo study.

PURPOSE: Hospital report cards are commonly produced using administrative data. The objective of this study was to determine the impact of unmeasured clinical data on the accuracy of hospitals' report cards. METHODS: Monte Carlo simulations were based on both administrative and detailed clinical data for patients hospitalized with an acute myocardial infarction in Ontario, Canada. Data were simulated such that the true performance of each hospital was known. Both clinical and administrative risk scores were randomly generated for each patient. The ability of hospital report cards to correctly identify hospitals that truly had higher than acceptable mortality was compared when both clinical and administrative data were used and when only administrative data were used. By using Monte Carlo simulations, we were able to incrementally increase the divergence between the 2 risk scores. RESULTS: In a wide range of settings, sensitivity and specificity of hospital report cards was only negligibly greater when both administrative and clinical data were used compared to when only administrative data were used. CONCLUSIONS: Unmeasured clinical data have at most a minor impact on the accuracy of cardiac hospital report cards.     

Why are health care report cards so bad (good)?

This paper provides a signaling-game theoretical foundation for empirically testing the effects of quality report cards in the U.S. health care industry. It shows that, when health care providers face an identical distribution of patient illness severities, the multidimensional measures in the existing report cards render them a mechanism that reveals the providers' qualities without causing them to select patients. However, non-identical patient type distributions between providers, attributed to the referring physician, may force the high-quality provider to shun patients in order to signal himself. Despite this imperfection, the existing report cards cause the minimum provider selection compared with alternative report mechanisms. Since the report cards not only may cause providers to select patients, but also cause patients to select providers, the single difference-in-differences estimates used in previous studies are not sufficient to indicate providers' selection behavior. In an updated empirical framework, a treatment effect shall be estimated once every period.     

Considering quality: engaging consumers to make better health care decisions

On May 19, 2005, Grantmakers In Health (GIH) convened nearly 50 grantmakers and leaders in the field of health care quality to discuss the challenges inherent in developing quality information, as well as how consumers use such information. This Issue Brief examines the potential of consumers to use information to select high-quality health services and to become drivers of quality improvement at the systems level. There are three broadly used mechanisms for connecting consumers to information: report cards on health plans and providers, decision support aides, and the Internet. Health foundations can play an important role in developing and disseminating quality information for consumers. They can support the development of quality indicators; assess the effectiveness of information technology, including the Internet; and fund advocacy efforts to ensure consumers have the information needed to make appropriate health care choices.     

Reporting the outcomes of care: effecting the transition from caring to comparing

Accountability in health care has taken on new dimensions with the drive to base contracting and provider of care selection upon data driven "report cards." The measurement and development of reportable outcomes are driving providers to move their organizations from a focus not only on "caring" but to one highly cognizant of "comparing" in order to maintain market position and meet regulatory requirements. This article defines the areas of organizational transition required for reporting, and profiles the actions taken by three health care providers moving to an organizational style ready for "comparative" competition.