Using patient reports to measure health care system performance

We developed a self-administered patient questionnaire that asks for data concerning the time to receive services (access to care), communication between providers (coordination of care), and follow up after tests and treatment (continuity of care). From these data, we construct rates of performance about the clinical management systems that support provision of these services. Rates of system performance are calculated for indicators using patients' responses to survey questions. These indicators add the number of patients reporting a problem of those patients who have encountered a particular clinical management system. Information derived from 3000 patient questionnaires is matched with data abstracted from health care medical records. The sensitivity and specificity of patient reports are being evaluated for all indicators classified as gold standards for medical records. Indicators considered gold standard items for patient reports are matched for agreement with any information contained in the medical record. Also, patient characteristics associated with accurate reporting is to be assessed using multivariate logistic regression models.     

The impact of evidence-based guideline dissemination for the assessment and treatment of major depression in a managed behavioral health care organization

This study tests whether a managed behavioral health care organization can influence adherence to practice guidelines for the treatment of major depression in a randomized trial of guideline dissemination. Guidelines were disseminated to mental health clinicians (N=443) under one of three conditions: (1) a general mailing of guidelines to clinicians, (2) a mailing in which guidelines were targeted to a patient starting treatment with the clinician, and (3) no mailing of guidelines. The results showed no effects of guideline dissemination as measured by self-report of patients and clinicians and through episode characteristics derived from claims data, despite sentinel effects. Results also showed high rates of clinician-reported guideline adherence that were not detected in the claims data, indicating significant undertreatment of depression. Results suggest that mental health systems must look to other dissemination strategies to improve adherence to standards of care and raise the performance of independent practicing clinicians     

Using a standardised patient assessment to measure professional attributes

INTRODUCTION: Professionalism is an important topic in medical education today. While much work has focused on defining professionalism and teaching medical students the appropriate interpersonal behaviours, relatively little research has looked at meaningful ways of assessing the relevant attributes. METHOD: The Educational Commission for Foreign Medical Graduates (ECFMG) clinical skills assessment (CSA) uses standardised patients (SPs) to evaluate the readiness of graduates of international medical schools to enter accredited graduate training programmes in the USA. Doctor interpersonal skills, including professional qualities such as rapport, are evaluated as part of the CSA. Attentiveness, attitude and empathy, all facets of professional behaviour, are specifically targeted as part of the assessment. RESULTS: To date, over 35 000 candidates have been assessed, encompassing more than 370 000 individual SP encounters. Based on a 1-year cohort of examinees, the reliability of the individual professionalism-related component scores ranged from 0.61 to 0.70. Doctors who had graduated from medical school more recently, or were younger, generally obtained higher ratings. Professional qualities were only marginally related to measures of basic science and clinical science proficiency. Female candidates were rated significantly higher than male candidates on all dimensions. CONCLUSIONS: While some professional behaviours are probably best measured using formats such as surveys, self-assessment and critical incident techniques, certain aspects of the domain can be reliably and validly measured in SP examinations.     

Use of a medical center's computerized health care database for notifiable disease surveillance.

The sensitivity of a medical center's inpatient and outpatient database to detect notifiable diseases was examined. Only 53 percent of inpatient and 7 percent of outpatient laboratory-confirmed cases of shigellosis, salmonellosis, giardiasis, and hepatitis were identified by an automated search for matching diagnosis codes. Reasons for lack of sensitivity include nonavailability of laboratory results at the time of diagnosis assignment, use of a standardized encounter form with limited preselected diagnosis codes, and pre-emptying of the infectious disease diagnosis by other diagnoses.     

Using willingness to pay to evaluate the implementation of Canada's residential radon exposure guideline

BACKGROUND: The objective of this investigation was to determine the effectiveness of Canada's residential radon exposure guideline in influencing individuals' health protection decisions. METHOD: Homeowners with known exposure levels in a high residential radon area (Winnipeg, Manitoba) were surveyed to document what they had done and spent to reduce their exposure to radon. The 507 respondents were then re-surveyed to elucidate their response to hypothetical scenarios. Logistic regression was used to model risk reduction decisions as a function of exposure and other explanatory variables. RESULTS: Homeowners were only likely to have taken action to reduce exposure at levels exceeding 1,100 Bq/m3, well above Canada's guideline of 800 Bq/m3. However, when informed of the guideline, respondents indicated they would act at exposures of 702 Bq/m3. INTERPRETATION: The Canadian residential radon exposure guideline, as it has been implemented, has not effectively prompted homeowner actions to reduce exposures to radon.     

Using a logic model to design and evaluate quality and patient safety improvement programs

Quality improvement programs often pose unique project management challenges, including multi-faceted interventions that evolve over time and teams with few resources for data collection. Thus, it is difficult to report methods and results. We developed a program to reduce central line-associated bloodstream infections (CLABSIs) and improve safety culture in intensive care units (ICUs). As previously reported, we worked with 103 Michigan ICUs to implement this program, and they achieved a 66% reduction in the median CLABSI rate and sustained the reduction. This success prompted the spread of this program to Spain, England, Peru and across the USA. We developed a logical framework approach (LFA) to guide project management; to incorporate the cultural, clinical and capacity variations among countries; and to ensure early alignment of the project's design and evaluation. In this paper, we describe the use of the LFA to systematically design, implement and evaluate large-scale, multi-faceted, quality improvement programs.     

Development and testing of a utility measure for major, unipolar depression (McSad)

Objective: To develop and test a direct utility measure (McSad) for major, unipolar depression. Methods: A depression specific, multi-attribute health state classification system was created; clinical validity was evaluated by experts using specially designed structured exercises; a cross-sectional survey was conducted to obtain directly measured utilities for depression health states. Setting: Tertiary care, university medical centre. Participants: Three psychiatrists, 3 psychiatric nurses and 3 social workers assessed depression health state clinical validity. Survey participants were referred by psychiatrists and consisted of 105 out-patients, currently in remission with at least one episode of major, unipolar depression in the past two years. Survey results: Respondent self-health state utility (mean and 95% confidence interval (CI)) was 0.79 (0.74–0.83). Utilities for hypothetical, untreated depression health states were: mild depression, 0.59 (0.55–0.62); moderate depression, 0.32 (0.29–0.34); severe depression, 0.04 (0.01–0.07). Fifty-six percent of respondents rated severe depression worse than being dead. Utilities for the hypothetical health states were not correlated with self-health utility. The intra-class correlation coefficient (ICC) was satisfactory for 13 of the 14 health states assessed. Conclusions: McSad was feasible and acceptable in patients with a history of major unipolar depression. The utilities for mild, moderate and severe untreated depression show the low health-related quality of life associated with depression. Initial assessments of test–retest reliability and validity yielded satisfactory results but further studies are needed to extend our knowledge of the measurement properties of McSad.     

Use of Datasus to evaluate psychiatric inpatient care patterns in Southern Brazil

OBJECTIVE: To describe the construction and testing of a routine to assess psychiatric hospitalizations in the Brazilian Health System based on its database (DATASUS), and to assess characteristics and trends of these hospitalizations. METHODS: Data were extracted from hospital admission authorizations in the state of Rio Grande do Sul, Southern Brazil, from 2000 to 2004. Data from 91,233 admissions were processed through a routine (syntaxes) using SPSS program and their reliability was tested. Hospitalization rates in general and psychiatric hospitals and main diagnoses were described, and trends were analyzed using polynomial regression models. RESULTS: Intra and inter-rater reliabilities were 100%. There was seen a trend of increasing hospitalization rates due to mood disorders and decreasing rates due to schizophrenia and organic disorders. Hospitalization rates due to substance use disorders remained stable. There was an increasing trend in the number of psychiatric hospitalizations in general hospitals with a 97.7% growth in the period studied. CONCLUSIONS: Routines proved to be reliable and feasible, suggesting the use of data from Hospital Information System database as a source of information for continuous evaluation of psychiatric hospitalizations in Brazilian Health System. Psychiatric hospitalization rates may have changed due to changes in the type of patients; diagnostic patterns, known as treatment-oriented diagnostic bias; and legislation.     

Comparison of international guideline programs to evaluate and update the Dutch program for clinical guideline development in physical therapy

Background  

Clinical guidelines are considered important instruments to improve quality in health care. Since 1998 the Royal Dutch Society for Physical Therapy (KNGF) produced evidence-based clinical guidelines, based on a standardized program. New developments in the field of guideline research raised the need to evaluate and update the KNGF guideline program.     

A method to create a standardized generic and condition-specific patient-reported outcome measure for patient care and healthcare improvement

Purpose

Patient-reported outcome measures (PROMs), which are generic or condition-specific, are used for a number of reasons, including clinical care, clinical trials, and in national-level efforts to monitor the quality of health care delivery. Creating PROMs that meet different purposes without overburdening patients, healthcare systems, providers, and data systems is paramount. The objective of this study was to test a generalizable method to incorporate condition-specific issues into generic PROM measures as a first step to producing PROMs that efficiently provide a standardized score. This paper outlines the method and preliminary findings focused on a PROM for osteoarthritis of the knee (OA-K).

Methods

We used a mixed-methods approach and PROMIS^® measures to test development of a combined generic and OA-K-specific PROM. Qualitative methods included patient focus groups and provider interviews to identify impacts of OA-K important to patients. We then conducted a thematic analysis and an item gap analysis: identified areas covered by existing generic PROMIS measures, identified “gap” areas not covered, compared gap areas to legacy instruments to verify relevance, and developed new items to address gaps. We then performed cognitive testing on new items and drafted an OA-K-specific instrument based on findings.

Results

We identified 52 existing PROMIS items and developed 24 new items across 14 domains.

Conclusions

We developed a process for creating condition-specific instruments that bridge gaps in existing generic measures. If successful, the methodology will create instruments that efficiently gather the patient’s perspective while allowing health systems, researchers, and other interested parties to monitor and compare outcomes over time, conditions, and populations.

     

Using a patient survey for marketing a professional health care practice

Small, private, professional health care practices are at a disadvantage when conducting market survey research because they cannot afford to employ or purchase the expensive specialized marketing skills of their larger competitors. The author describes a method that small private practices can use to conduct patient marketing surveys. Survey findings are reported and examples are provided of how the results influenced subsequent marketing decisions. Suggestions are offered to help ensure the success of similar studies in other practices.     

Development of a measure of attitude toward nutrition in patient care

BACKGROUND: Development of reliable measures of medical student and resident attitudes about nutrition in patient care is needed before the effects of educational interventions or clinical experience can be gauged. This report describes the systematic development of a measure of attitude toward nutrition in patient care. It presents evidence about scale reliability and the absence of response bias that endorses the trustworthiness of data from the measure. METHODS: An eight-step attitude scale development procedure was used to create the Nutrition In Patient care Survey (NIPS). Data from five samples of first- and second-year medical students and first-year medical residents were subjected to factor analysis (PA2, varimax rotation), reliability analyses, and statistical analyses to test for demographic bias in the attitude data. RESULTS: A 45-item attitude measure was developed that contains five subscales derived from the factor analysis: (1) nutrition in routine care (NRC, 8 items); (2) clinical behavior (CB, 20 items); (3) physician-patient relationship (PPR, 8 items); (4) patient behavior/motivation (PBM, 3 items); and (5) physician efficacy (PE, 6 items). Each subscale yields reliable data in terms of internal consistency (alpha coefficients) and stability (test-retest reliability). Medical student and resident demographic variables have negligible influence on attitude scores. DISCUSSION: The NIPS subscales yield reliable data that can be used to assess outcomes in evaluation research on educational or clinical interventions or to predict patient care practices. Systematic attitude scale development increases the likelihood that the resulting measures will produce useful, trustworthy data.     

Using discrete choice experiments to estimate a preference-based measure of outcome--an application to social care for older people

This paper reports the results of a study that used discrete choice experiment (DCE) methodology to estimate quality weights for a of social care outcome measure. To reflect different states of need, a five-dimensional profile measure was developed. Experimental design techniques were used to derive a sample of states for which preferences were elicited. The DCE approach was employed to elicit values and regression techniques used to estimate a model that could predict index scores for all 243 possible outcomes. The utility model, estimated on the basis of 297 responses, had good test-retest reliability and converged with preferences elicited from a rating exercise.