Lottery incentives did not improve response rate to a mailed survey: a randomized controlled trial

OBJECTIVES: Our study aimed to examine the effect of an instant lottery ticket incentive on the response rate to a mailed questionnaire in a population of trauma patients. STUDY DESIGN AND SETTING: A randomized controlled trial at a major trauma center with 728 patients randomized into 2 groups prior to mailing of a questionnaire. One group of patients (Group A) had a sentence inserted into the cover letter stating that they would receive a $4 instant lottery ticket upon receipt of a completed questionnaire; Group B did not have an incentive. The response rate for both groups was measured after the initial mailing and at the end of the study. The results were analyzed using the X2 test to compare 2 proportions and a P value of .05 was considered significant. RESULTS: The early response rate in Group A was lower than in Group B, and the response rates for both groups were similar at final follow-up. The differences at both time periods were not statistically significant. CONCLUSION: The use of an instant lottery ticket incentive did not improve the response rate to a mailed questionnaire.     

Prepayment was superior to postpayment cash incentives in a randomized postal survey among physicians

OBJECTIVE: Improving response rates, particularly among physicians, is important to minimize nonresponder bias and increase the effective sample size in epidemiologic research. We conducted a randomized trial to examine the impact of prepayment vs. postpayment incentives on response rates. STUDY DESIGN AND SETTING: Self-completion postal questionnaires were mailed to 949 physicians who were respondents to an earlier survey and representative of the general physician population in Hong Kong. These physicians were randomly allocated to receive a HK dollar 20 cash prepayment incentive that accompanied the survey (n=474) or a postpayment reward of the same amount on receipt of the completed questionnaire (n=475). RESULTS: The final prepayment response rate was 82.9%, compared with 72.5% in the postpayment arm (P < .001). Of the eight alternative incentive and follow-up strategies evaluated, three lie on the efficiency frontier (i.e., not dominated), including postpayment with three mailings at HK dollar 42.7, prepayment with three mailings at HK dollar 66.5 and prepayment with three mailings and telephone follow-up at HK dollar 112.1 per responder recruited (US dollar 1=HK dollar 7.8). CONCLUSION: The findings demonstrate that prepayment cash incentives are superior to postpayment of the equivalent amount in improving response rates among a representative sample of Hong Kong physicians. Further research should concentrate on confirming the generalizability of these findings in other health care occupation groups and settings.     

Questionnaire color and response rates to mailed surveys. A randomized trial and a meta-analysis

The authors conducted a randomized trial in Geneva, Switzerland, to assess whether response rates to a mailed survey could be increased by printing the questionnaire on green paper. The authors also conducted a meta-analysis of 10 experimental studies that tested the effect of colored questionnaires on response rates. The randomized trial showed no effect (relative risk of responding [RR] = 1.00). The meta-analysis showed that mailing questionnaires on pink paper increased response rates by 12% (RR = 1.12, 95% confidence interval = 1.01 to 1.25, p = 0.04). Other colors had no statistically significant effect (blue: RR = 1.03, p = 0.49; green: RR = 1.02, p = 0.23; yellow: RR = 0.96, p = 0.30). Overall, using colored instead of white paper had no effect (RR = 1.02, p = 0.17). Thus, printing questionnaires on colored paper does not substantially increase response rates in surveys, except for pink paper.     

The cost-effectiveness of cash versus lottery incentives for a web-based, stated-preference community survey

We present the results of a randomized experiment to test the effectiveness and cost-effectiveness of response incentives for a stated-preference survey of a general community population. The survey was administered using a mixed-mode approach, in which community members were invited to participate using a traditional mailed letter using contact information for a representative sample of the community; but individuals completed the survey via the web, which exploited the advantages of electronic capture. Individuals were randomized to four incentive groups: (a) no incentive, (b) prepaid cash incentive ($2), (c) a low lottery (10 prizes of $25) and (d) a high lottery (2 prizes of $250). Letters of invitation were mailed to 3,000 individuals. In total, 405 individuals (14.4%) contacted the website and 277 (9.8%) provided complete responses. The prepaid cash incentive generated the highest contact and response rates (23.3 and 17.3%, respectively), and no incentive generated the lowest (9.1 and 5.7%, respectively). The high lottery, however, was the most cost-effective incentive for obtaining completed surveys: compared with no incentive, the incremental cost-effectiveness ratio (ICER) per completed survey for high lottery was $13.89; for prepaid cash, the ICER was $18.29. This finding suggests that the preferred response incentive for community-based, stated-preference surveys is a lottery with a small number of large prizes.     

Randomized trial of financial incentives and delivery methods for improving response to a mailed questionnaire

In a follow-up study, only 64% of 126,628 US radiologic technologists completed a questionnaire during 1994-1997 after two mailings. The authors conducted a randomized trial of financial incentives and delivery methods to identify the least costly approach for increasing overall participation. They randomly selected nine samples of 300 nonresponders each to receive combinations of no, 1.00 US dollar, 2.00 US dollars, and 5.00 US dollars cash or check incentives delivered by first-class mail or Federal Express. Federal Express delivery did not achieve greater participation than first-class mail (23.2% vs. 23.7%). In analyses pooled across delivery methods, the response was significantly greater for the 2.00 US dollar bill (28.9%, 95% confidence interval (CI): 25.2, 32.7; p < 0.0001), 5.00 US dollars check (27.5%, 95% CI: 22.5, 33.0; p = 0.0001), 1.00 US dollar bill (24.6%, 95% CI: 21.2, 28.3; p = 0.0007), and 2.00 US dollars check (21.8%, 95% CI: 18.5, 25.3; p = 0.02) compared with no incentive (16.6%, 95% CI: 13.7, 19.9). The response increased significantly with increasing incentive amounts from 0.00 to 2.00 US dollars cash (p trend < 0.0001). The 2.00 US dollar bill achieved a 30% greater response than did a 2.00 US dollars check (p = 0.005). For incentives sent by first-class mail, the 5.00 US dollars check yielded 30% greater participation than did the 2.00 US dollars check (p = 0.07). A 1.00 US dollar bill, chosen instead of the 2.00 US dollars bill because of substantially lower overall cost and sent by first-class mail to the remaining 42,717 nonresponders, increased response from 64% to 72%.     

Postal surveys of physicians gave superior response rates over telephone interviews in a randomized trial

BACKGROUND AND OBJECTIVES: To compare general practitioner (GP) response to a telephone interview with response to a postal survey with three reminders in a randomized controlled trial. METHODS: GPs were randomly assigned to either a telephone interview or a postal survey. GPs in the telephone group were mailed a letter of invitation and asked to undertake a telephone interview. GPs in the postal group were mailed a letter of invitation and questionnaire. Non-responders were sent up to three reminders, the final by registered post. Response rates were calculated for each group. RESULTS: 416 GPs were randomized to the telephone interview and 451 to the postal survey. Eighty-six in the telephone group and 30 in the postal were ineligible. One hundred thirty-four GPs completed the telephone interview with a response rate of 40.6% (95% confidence interval [CI]: 35.3%, 46.1%). Two hundred fifty-two GPs completed the postal survey with a response rate of 59.9% (95%CI: 55.0%, 64.6%). The difference in response was 19.3% (95%CI: 12.2%, 26.3%). CONCLUSIONS: These results show that postal surveys with three reminders can have superior response rates compared with a telephone interview.     

Reducing occupational psychological distress: a randomized controlled trial of a mailed intervention

There are increasing levels of psychological distress among general practitioners (GPs). The purpose of this study is to evaluate the effectiveness of a mailed intervention to reduce distress among 'at-risk' GPs. A questionnaire was sent to 1356 GPs from eight Divisions of General Practice. Out of 819 (60%) who responded, 233 GPs were recruited with scores indicative of psychological distress. These GPs were randomized to intervention (n = 120) or control (n = 113). The intervention consisted of a simple letter feeding back and interpreting the psychological score together with a self-help sheet. During the study, an educational program was offered to GPs by Divisions of General Practice. The main outcome measure used was changes in psychological distress (General Health Questionnaire 12) score after 3 months. Significance was set at P < 0.05. Initial analysis of the data showed borderline significance (P = 0.05). However, analysis of the data post hoc excluding GPs who participated in the educational program showed a significant reduction in psychological distress (P = 0.03). It appears that there may have been a dilution of the intervention effect. Mailed interventions are a cost-effective way of reaching at-risk GPs and may contribute to a reduction in psychological morbidity.     

Online smoking cessation program for Korean Americans: Randomized trial to test effects of incentives for program completion and interim surveys

IntroductionSmoking is prevalent among Korean American men. Quitting is Winning, an Internet-based, cognitive-behavioral smoking cessation program, was developed using community-based participatory research principles.MethodsA randomized controlled trial was used to evaluate whether participants were more likely to complete the program and quit smoking at 6-months of follow-up with additional reinforcement. The main outcomes were the proportion of participants who completed the online program and the proportion who quit smoking for at least 30 days, 26 weeks after enrollment, among those randomized into the high-reinforcement (HR) condition compared with those in the low-reinforcement (LR) condition.ResultsThe study achieved a final enrollment of 403 participants including 56 women. Program completion was greater for the HR as compared to the LR condition (17% vs. 10%, p = .035). There was no significant difference in 30-day smoking cessation (intent-to-treat [ITT]) between the HR and LR conditions (9% vs. 8%, ns). Smoking cessation was greater among program completers as compared to those who did not complete the program (28% vs. 5%, p < .001).ConclusionsThe addition of interim surveys and financial incentives for interim survey completion and program completion significantly increased the likelihood of program completion. Moreover, program completers were significantly more likely to quit smoking. Although smoking cessation rates did not significantly differ between the HR and LR conditions, the results suggest that future studies should explore the efficacy of larger financial incentives for program completion (Clinical Trial #NCT02584127).     

Physician participation in research surveys. A randomized study of inducements to return mailed research questionnaires

The authors randomly selected 400 physicians from a population of 1,545 practicing physicians providing follow-up care to patients who received bone marrow or blood stem cell transplants at the Fred Hutchinson Cancer Research Center to determine interest in receiving Internet-based transplant information. In a two-factor completely randomized factorial design, the 400 physicians were assigned to receive mailed surveys with either no compensation or a $5 check and either no follow-up call or a follow-up call 3 weeks after mailing. Overall, 51.5% of the physicians returned the mailed surveys. Comparison of logit models showed that inclusion of a $5 check in the mailer significantly (p = .016) increased the probability of returning the surveys (57.5% vs. 45.5%). In contrast, the telephone follow-up had no overall effect. The authors concluded a modest financial reward can significantly improve physician response rates to research surveys but a telephone follow-up may be inefficient and even ineffective.     

Effects of mailed advice on stress reduction among employees in Japan: a randomized controlled trial.

We conducted a randomized controlled trial (RCT) to examine the effects of mailed advice on reducing psychological distress, blood pressure, serum lipids, and sick leave of workers employed in a manufacturing plant in Japan. Those who indicated higher psychological distress (defined as having GHQ scores of three or greater) in the baseline questionnaire survey (n = 226) were randomly assigned to an intervention group or a control group. Individualized letters were sent to the subjects of the intervention group, informing them of their stress levels and recommending an improvement in daily habits and other behaviors to reduce stress. Eighty-one and 77 subjects in the intervention and control groups, respectively, responded to the one-year follow-up survey. No significant intervention effect was observed for the GHQ scores, blood pressure, serum lipids, or sick leave (p > 0.05). The intervention effect was marginally significant for changes in regular breakfasts and daily alcohol consumption (p = 0.09). The intervention effect was marginally significant for the GHQ scores among those who initially did not eat breakfast regularly (p = 0.06). The study suggests that only sending mailed advice is not an effective measure for worksite stress reduction. Mailed advice which focuses on a particular subgroup (e.g., those who do not eat breakfast regularly) may be more effective.     

The effects of lottery incentive and length of questionnaire on health survey response rates: a randomized study.

Maximizing the response rate of self-administered questionnaires is key in survey research. We aimed to evaluate the effects of lottery incentive and length of questionnaire on health survey response rates when used in isolation or combined. A random sample of 440 residents in Western Sydney, Australia was randomly allocated to four equal groups to receive or not receive an instant lottery ticket and a long (seven page) or short (one page) questionnaire. The overall response rate was 71.8%. The final response rates were higher among those receiving the short, rather than the long, questionnaire (75.6% versus 68.2%) (P = 0.08); and among those receiving the lottery incentive compared with those not receiving the incentive (75% versus 68.2%) (P = 0.09). By logistic regression analysis, the success of obtaining a completed questionnaire without any follow-up reminders was significantly associated with the lottery incentive but not the questionnaire length (P = 0.03 and P = 0.54, respectively). The difference between lottery and no lottery groups decreased gradually during the follow-up. A lottery incentive is associated with an increased response after the first mailing. A small up-front cost for a lottery ticket may be worthwhile, since it can save further costs by obviating the need for repeated follow-ups.     

Meta-analysis of randomised trials of monetary incentives and response to mailed questionnaires

STUDY OBJECTIVE: To quantify the increase in mailed questionnaire response attributable to a monetary incentive. DESIGN: A systematic search for randomised controlled trials of monetary incentives and mailed questionnaire response was conducted. For each trial identified, logistic regression was used to estimate the odds ratio for response per 0.01 dollars incentive increase. Odds ratios were pooled in a series of random effect meta-analyses stratified by the minimum and maximum amounts offered. Piecewise logistic regression was used to estimate the odds ratio for response per $0.01 increase given in each of five incentive ranges. SETTING: Populations in several developed countries, predominantly the USA. PARTICIPANTS: 85,671 randomised participants from 88 trials. MAIN RESULTS: The pooled odds ratios for response per 0.01 dollars incentive decreased monotonically as the maximum amount of incentive offered increased. The piecewise logistic regression model estimated that for incentive amounts up to 0.50 dollars, each additional 0.01 dollars increased the odds of response by about 1% (pooled OR = 1.012, 95%CI 1.007 to 1.016). The effects on response above 0.50 dollars were smaller and decreased monotonically in the ranges: 0.50-0.99 dollars, 1-1.99 dollars, 2-4.99 dollars, 5.00 dollars and over, but remained statistically significant up to 5 dollars. CONCLUSIONS: This meta-analysis of the best available evidence shows that monetary incentives increase mailed questionnaire response. Researchers should include small amounts of money with mailed questionnaires rather than give no incentive at all.     

Financial incentives for physicians: The Quebec experience

This paper presents an empirical investigation of physician labour supply, based on a two-stage budgeting model, drawing on an analogy with consumer theory. Physicians' trade-offs between income and leisure constitute the first stage of the decision-making process. In turn, choices are made in the second stage concerning the choice of particular activities (hospital versus office care, for example) or procedures (ordinary versus complete medical examinations), given the total medical care activity chosen in the first stage. The objective of the study is to identify physicians' responses to exogenous shocks in the remuneration system. The focus of analysis is shifted away from the identification of Supply-Induced Demand (SID) to a more pragmatic analysis of some of the determinants of physicians' choices. The study uses monthly activity data on a panel of 677 Québec GPs between 1977 and 1983. Quantity adjustments and drifts to more complex (and therefore better paid) procedures are evidenced, mainly in response to a fifteen month tariff-freeze. Physicians' ability to control their own work loads is also documented, both in terms of timing and level of complexity, and expenditure caps (in the form of an individual ceiling on GPs' quarterly gross income) are found to be effective at curbing high activity rates.     

A randomised controlled trial to determine the effect on response of including a lottery incentive in health surveys [ISRCTN32203485]

Background

Postal questionnaires are an economical and simple method of data collection for research purposes but are subject to non-response bias. Several studies have explored the effect of monetary and non-monetary incentives on response. Recent meta-analyses conclude that financial incentives are an effective way of increasing response rates. However, large surveys rarely have the resources to reward individual participants. Three previous papers report on the effectiveness of lottery incentives with contradictory results. This study aimed to determine the effect of including a lottery-style incentive on response rates to a postal health survey.

Methods

Randomised controlled trial. Setting: North and West Birmingham. 8,645 patients aged 18 or over randomly selected from registers of eight general practices (family physician practices). Intervention: Inclusion of a flyer and letter with a health questionnaire informing patients that returned questionnaires would be entered into a lottery-style draw for £100 of gift vouchers. Control: Health questionnaire accompanied only by standard letter of explanation. Main outcome measures: Response rate and completion rate to questionnaire.

Results

5,209 individuals responded with identical rates in both groups (62.1%). Practice, patient age, sex and Townsend score (a postcode based deprivation measure) were identified as predictive of response, with higher response related to older age, being female and living in an area with a lower Townsend score (less deprived).

Conclusion

This RCT, using a large community based sample, found that the offer of entry into a lottery style draw for £100 of High Street vouchers has no effect on response rates to a postal health questionnaire.     

The effect of economic incentives on the education and distribution of physicians: a review.

This paper examines the available evidence on the impact of economic factors on the specialty and locational choices of physicians. Economic variables which influence the "rate of return" to the physician (profitability in relation to training costs) to alternative specialties and locational decisions include average yearly income, hours of work, price for each health service and training costs. The findings of the review indicate that the rate of return to specialty training varies substantially among specialties. Rates of return to training in surgery and radiology are nearly three times that of other medical specialties. These rates of return differences are shown to have a small, albeit significant, effect on a physician's specialty as well as location choices. Furthermore, there is a positive relationship between the mean fees of physicians and the physician population ratio in an area, i.e., areas with more physicians have higher fees. Confounding the relationship between economic variables and specialty and locational choice is the fact that physicians may have substantial amounts of market power and can themselves influence the price of their services. Thus, the influence of reimbursement policies to alter the distribution of physicians may be less effective because physicians may have the ability to influence and alter the level of income and rate of return to training.