Using Vignettes to Compare the Quality of Clinical Care Variation in Economically Divergent Countries

Objective. To determine whether clinical vignettes can measure variations in the quality of clinical care in two economically divergent countries.
Data Source/Study Setting. Primary data collected between February 1997 and February 1998 at two Veterans Affairs facilities in the United States and four government-run outpatient facilities in Macedonia.
Study Design. Randomly selected, eligible Macedonian and U.S. physicians (>97 percent participation rate) completed vignettes for four common outpatient conditions. Responses were judged against a master list of explicit quality criteria and scored as percent correct.
Data Collection/ Extraction. An ANOVA model and two-tailed t-tests were used to compare overall scores by case, study site, and country.
Principal Findings. The mean score for U.S. physicians was 67 percent (+/−11 percent) compared to 48 percent (+/−11 percent) for Macedonian physicians. The quality of clinical practice, which emphasizes basic skills, varied greatly in both sites, but more so in Macedonia. However, the top Macedonian physicians in all sites approached or—in one case—exceeded the median score in the U.S. sites.
Conclusions. Vignettes are a useful method for making cross-national comparisons of the quality of care provided in very different settings. The vignette measurements revealed that some physicians in Macedonia performed at a standard comparable to that of their counterparts in the United States, despite the disparity of the two health systems. We infer that in poorer countries, policy that promotes improvements in the quality of clinical practice—not just structural inputs—could lead to rapid improvements in health.     

The development and piloting of an instrument to measure care staff attitudes towards child self-harm

Abstract

Child self-harm has been identified as an important issue within paediatric care, but the attitudes of care staff towards the topic have received limited investigation. The aim of this paper is the justification and development of an instrument to measure care staff attitudes towards child self-harm. By identifying the salient features of the phenomenon and incorporating them into a means of data collection, it is hoped that a more realistic picture of the clinical environment emerges. The main elements of the instrument are introduced in turn and its initial piloting is described. A questionnaire was developed based on a series of hypothetical case vignettes and subjected to a number of tests. The results of the pilot test indicated that the instrument shows high levels of reliability and would therefore prove useful in eliciting staff attitudes.     

An evaluation of vignettes for predicting variation in the quality of preventive care

OBJECTIVE: Clinical vignettes offer an inexpensive and convenient alternative to the benchmark method of chart audits for assessing quality of care. We examined whether vignettes accurately measure and predict variation in the quality of preventive care. DESIGN: We developed scoring criteria based on national guidelines for 11 prevention items, categorized as vaccine, vascular-related, cancer screening, and personal behaviors. Three measurement methods were used to ascertain the quality of care provided by clinicians seeing trained actors (standardized patients; SPs) presenting with common outpatient conditions: 1) the abstracted medical record from an SP visit; 2) SP reports of physician practice during those visits; and 3) physician responses to matching computerized case scenarios (clinical vignettes). SETTING: Three university-affiliated (including 2 VA) and one community general internal medicine clinics. PATIENTS/PARTICIPANTS: Seventy-one randomly selected physicians from among eligible general internal medicine residents and attending physicians. MEASUREMENTS AND MAIN RESULTS: Physicians saw 480 SPs (120 at each site) and completed 480 vignettes. We calculated the proportion of prevention items for each visit reported or recorded by the 3 measurement methods. We developed a multiple regression model to determine whether site, training level, or clinical condition predicted prevention performance for each measurement method. We found that overall prevention scores ranged from 57% (SP) to 54% (vignettes) to 46% (chart abstraction). Vignettes matched or exceeded SP scores for 3 prevention categories (vaccine, vascular-related, and personal behavior). Prevention quality varied by site (from 40% to 67%) and was predicted similarly by vignettes and SPs. CONCLUSIONS: Vignettes can measure and predict prevention performance. Vignettes may be a less costly way to assess prevention performance that also controls for patient case-mix. This research was funded by grant 11R98118-1 from the Veterans Affairs Health Service Research and Development Service, Washington DC. Dr. Peabody was also a recipient of a Senior Research Associate Career Development Award from 1998–2001 from the Department of Veterans Affairs.     

A vignette study to examine health care professionals' attitudes towards patient involvement in error prevention

Background Various authorities recommend the participation of patients in promoting patient safety, but little is known about health care professionals' (HCPs') attitudes towards patients' involvement in safety‐related behaviours. Objective To investigate how HCPs evaluate patients' behaviours and HCP responses to patient involvement in the behaviour, relative to different aspects of the patient, the involved HCP and the potential error. Design Cross‐sectional fractional factorial survey with seven factors embedded in two error scenarios (missed hand hygiene, medication error). Each survey included two randomized vignettes that described the potential error, a patient's reaction to that error and the HCP response to the patient. Setting Twelve hospitals in Switzerland. Participants A total of 1141 HCPs (response rate 45%). Measurements Approval of patients' behaviour, HCP response to the patient, anticipated effects on the patient–HCP relationship, HCPs' support for being asked the question, affective response to the vignettes. Outcomes were measured on 7‐point scales. Results Approval of patients' safety‐related interventions was generally high and largely affected by patients' behaviour and correct identification of error. Anticipated effects on the patient–HCP relationship were much less positive, little correlated with approval of patients' behaviour and were mainly determined by the HCP response to intervening patients. HCPs expressed more favourable attitudes towards patients intervening about a medication error than about hand sanitation. Conclusions This study provides the first insights into predictors of HCPs' attitudes towards patient engagement in safety. Future research is however required to assess the generalizability of the findings into practice before training can be designed to address critical issues.     

Does reporting heterogeneity bias the measurement of health disparities?

Heterogeneity in reporting of health by socio-economic and demographic characteristics potentially biases the measurement of health disparities. We use anchoring vignettes to identify socio-demographic differences in the reporting of health in Indonesia, India and China. Homogeneous reporting by socio-demographic group is rejected and correcting for reporting heterogeneity tends to reduce slightly estimated disparities in health by education (not China) and to increase those by income. But the method does not reveal substantial reporting bias in measures of health disparities.