Provider attitudes toward pay-for-performance programs: development and validation of a measurement instrument

OBJECTIVE: To develop an instrument for assessing physician attitudes toward quality incentive programs, and to assess its reliability and validity. DATA SOURCES: Study involved primary data collection. A 40-item paper and pencil survey of primary care physicians in Rochester, New York, and Massachusetts was conducted between May 2004 and December 2004. Seven-hundred and ninety-eight completed questionnaires were received, representing a response rate of 32 percent (798/2,497). STUDY DESIGN: Based on an extensive review of the literature and discussions with experts in the field, we developed a conceptual framework representing the features of pay-for-performance (P4P) programs hypothesized to affect physician behavior in that context. A draft questionnaire was developed based on that conceptual model and pilot tested in three groups of physicians. The questionnaire was modified based on the physician feedback, and the revised version was distributed to 2,497 primary care physicians affiliated with two of the seven sites participating in Rewarding Results, a national evaluation of quality target and financial incentive programs. DATA COLLECTION: Respondents were randomly divided into a derivation and a validation sample. Exploratory factor analysis was applied to the responses of the derivation sample. Those results were used to create scales in the validation sample, and these were then subjected to multitrait analysis (MTA). One scale representing physicians' perception of the impact of P4P on their clinical practice was regressed on the other scales as a test of construct validity. PRINCIPAL FINDINGS: Seven constructs were identified and demonstrated substantial convergent and discriminant validity in the MTA: awareness and understanding, clinical relevance, cooperation, unintended consequences, control, financial salience, and impact. Internal consistency reliabilities (Cronbach's alpha coefficients) ranged from 0.50 to 0.80. A statistically significant 25 percent of the variation in perceived impact was accounted for by physician perceptions of the other six characteristics of P4P programs. CONCLUSIONS: It is possible to identify and measure the key salient features of P4P programs using a valid and reliable 26-item survey. This instrument may now be used in further studies to better understand the impact of P4P programs on physician behavior.