The quality of the quality indicator of pain derived from the minimum data set

OBJECTIVE: To examine facility variation in data quality of the level of pain documented in the minimum data set (MDS) as a function of level of hospice enrollment in nursing homes (NHs). DATA SOURCE: Clinical assessments on 3,469 nonhospice residents from 178 NHs were merged with On-line Survey Certification and Reporting data of 2000, Medicare Claims data of 2000 and the MDS of 2000-2002. STUDY DESIGN: Using the same assessment protocol, NH staff and study nurses independently assessed 3,469 nonhospice residents. Study nurses' assessments being gold standard, we quantified and compared quality of NH staff's pain rating across NHs with high, medium, or low hospice use. Multilevel models were built to assess the effect of NH hospice use levels on the occurrence of false positive (FP) and false negative (FN) errors in NH-rated "severe pain." PRINCIPAL FINDINGS: Of 178 NHs, 25 had medium and 41 high hospice use. NHs with higher hospice use had lower sensitivities. In multilevel analysis, we found a significant facility-level variation in the probability of FP and FN errors in facility-rated "severe pain." Resident characteristics only explained 4 and 0 percent of the facility variation in FP and FN, respectively; characteristics and locations (state) of NHs further explained 53 and 52 percent of the variance. After controlling for resident and NH characteristics, staff in NHs with medium or high hospice use were less likely to have FP or FN errors in their MDS documentation of pain than were staff in NHs with low or no hospice use. CONCLUSIONS: The examination of data quality of pooled MDS data from multiple NHs is insufficient. Multilevel analysis is needed to elucidate sources of heterogeneity in the quality of MDS data across NHs. Facility characteristics, e.g., hospice use or NH location, are systematically associated with overrated/underrated pain and may bias pain quality indicator (QI) comparisons. To ensure the integrity of QI comparison in the NH setting, the government may need to institute regular audits of MDS data quality.