A prenotification letter increased initial response,whereas sender did not affect response rates

ObjectiveTo find ways to improve response rates of medical and health surveys. We investigated whether a prenotification letter instead of a second reminder and varying senders of the questionnaires would affect response rates.Study Design and SettingWe present the results of two studies. In the first study, four groups were compared that either received a prenotification letter (group 1 and 2) or a second reminder letter (group 3 and 4); received the questionnaire from either a research institute (group 1 and 3) or a health insurance company (HIC; group 2 and 4). In the second study, we compared two groups that received the questionnaire sent by either a HIC or a hospital. Response rates, response speed, respondent characteristics, item nonresponse, and mean scores on quality aspects and global ratings were compared.ResultsResponse rates did not differ significantly between groups. Prenotification groups returned their questionnaires faster. No other significant differences were found for response speed, respondent characteristics, item nonresponse, or mean scores.ConclusionA prenotification letter does only increase initial response speed and does not increase total response rates. A prenotification letter should be considered when quick response is desirable. Varying senders had no effect on response rates.     

The effects of cash and lottery incentives on mailed surveys to physicians: a randomized trial

Low response rates, especially among physicians, are a common problem in mailed survey research. We conducted a randomized trial to examine the effects of cash and lottery incentives on response rates. A total of 4,850 subjects were randomized to one of three interventions accompanying a mailed survey-no incentive (n = 1,700), cash payment [three levels of Hong Kong dollars (HKD) $10, $20, and $40; N = 50 in each subgroup], or entry into a lottery (three levels of HKD$1,000, $2,000, and $4,000; N = 1,000 in each subgroup) on receipt of the completed questionnaire. The response rates were higher among those offered incentives than those without (19.8% vs. 16.8%, P =.012). Cash was the more effective incentive compared to lottery (27.3% vs. 19.4%, P =.017). Response also increased substantially between the first and second mailings (14.2% vs. 18.8%, P >.001). In addition, those with specialist qualifications were more willing to participate in mailed surveys. We found no significant differences in response outcomes among the various incentive arms. Cash reward at the $20 level was the most cost-effective intervention, in terms of cost per responder. Further systematic examination of the effects of different incentive strategies in epidemiologic studies should be encouraged.     

Effect of incentives and mailing features on online health program enrollment.

BACKGROUND: With the growing use of Internet-based interventions, strategies are needed to encourage broader participation. This study examined the effects of combinations of monetary incentives and mailing characteristics on enrollment, retention, and cost effectiveness for an online health program. METHODS: In 2004, a recruitment letter was mailed to randomly selected Midwestern integrated health system members aged 21-65 and stratified by gender and race/ethnicity; recipients were randomly pre-assigned to one of 24 combinations of incentives and various mailing characteristics. Enrollment and 3-month retention rates were measured by completion of online surveys. Analysis, completed in 2005, compared enrollment and retention factors using t tests and chi-square tests. Multivariate logistic regression modeling assessed the probability of enrollment and retention. RESULTS: Of 12,289 subjects, 531 (4.3%) enrolled online, ranging from 1% to 11% by incentive combination. Highest enrollment occurred with unconditional incentives, and responses varied by gender. Retention rates ranged from 0% to 100%, with highest retention linked to higher-value incentives. The combination of a $2 bill prepaid incentive and the promise of $20 for retention (10% enrollment and 71% retention) was optimal, considering per-subject recruitment costs ($32 enrollment, $70 retention) and equivalent enrollment by gender and race/ethnicity. CONCLUSIONS: Cash incentives improved enrollment in an online health program. Men and women responded differently to mailing characteristics and incentives. Including a small prepaid monetary incentive ($2 or $5) and revealing the higher promised-retention incentive was cost effective and boosted enrollment.     

The effects of variations in mode of delivery and monetary incentive on physicians' responses to a mailed survey assessing STD practice patterns

High response rates from physicians are key to obtaining valid and generalizable data regarding their sexually transmitted disease (STD) diagnosis, treatment, and control practices. A factorial (3 x 2) study was designed using varying cash incentives ($0, $15, $25) and delivery modes (Federal Express, U.S. mail). Surveys, with three follow-up mailings, were sent to a national probability sample of 311 physicians in OB-GYN, family practice, internal and emergency medicine, and pediatrics specialties. Overall, 156 physicians returned completed surveys (56% overall response rate). Significant effects for incentive level (F = 28.2, df = 2, p < .01) and delivery mode (F = 4.1, df = 1, p < .05) existed. Highest response was among physicians in the $25-FedEx condition (81%). High response rates from busy practicing physicians can be achieved if surveys are relevant to clinical practice, sponsored by a reputable organization (the Centers for Disease Control and Prevention), include a monetary incentive, and are delivered by courier.     

The cost-effectiveness of cash versus lottery incentives for a web-based, stated-preference community survey

We present the results of a randomized experiment to test the effectiveness and cost-effectiveness of response incentives for a stated-preference survey of a general community population. The survey was administered using a mixed-mode approach, in which community members were invited to participate using a traditional mailed letter using contact information for a representative sample of the community; but individuals completed the survey via the web, which exploited the advantages of electronic capture. Individuals were randomized to four incentive groups: (a) no incentive, (b) prepaid cash incentive ($2), (c) a low lottery (10 prizes of $25) and (d) a high lottery (2 prizes of $250). Letters of invitation were mailed to 3,000 individuals. In total, 405 individuals (14.4%) contacted the website and 277 (9.8%) provided complete responses. The prepaid cash incentive generated the highest contact and response rates (23.3 and 17.3%, respectively), and no incentive generated the lowest (9.1 and 5.7%, respectively). The high lottery, however, was the most cost-effective incentive for obtaining completed surveys: compared with no incentive, the incremental cost-effectiveness ratio (ICER) per completed survey for high lottery was $13.89; for prepaid cash, the ICER was $18.29. This finding suggests that the preferred response incentive for community-based, stated-preference surveys is a lottery with a small number of large prizes.     

An advance letter did not increase the response rates in a telephone survey: a randomized trial

ObjectiveTo test the impact of an advance letter on response and cooperation rates in a nationwide telephone survey, given previous inconsistent results.Study Design and SettingWithin the context of a larger telephone survey, 1,000 Australian households were randomly selected to take part in this trial. Half were randomly allocated to receive an advance letter, whereas the remainder did not receive any advance communication. Response and cooperation rates were compared between the two groups.ResultsA total of 244 interviews were completed, 134 of which were with households that had been sent an advance letter. Intention-to-treat analysis revealed no significant difference in response between those who had received a letter and those who had not (26.8% vs. 22.0%, respectively). In addition, there was no significant difference between the groups in terms of either cooperation (78.4% vs. 79.7%) or response rate (56.3% vs. 57.9%), and no clear differences emerged in terms of the demographic characteristics of the two groups.ConclusionAn advance letter was not seen to be effective in increasing response or cooperation rates in a nationwide telephone survey. Researchers should consider alternative methods of increasing participation in telephone surveys.     

Can incentives undermine intrinsic motivation to participate in epidemiologic surveys?

Response rates to surveys are decreasing. The purpose of this study was to evaluate the use of lottery tickets as incentives in an epidemiologic control group. A self-administered questionnaire was sent to parents in the municipality of Stockholm, Sweden, who were to be used as a control group in a study addressing stress in parents of children with cancer. A stratified random sample of 450 parents were randomized into three incentive groups: (a) no incentive; (b) a promised incentive of one lottery ticket to be received upon reply; (c) a promised incentive of one lottery ticket to be received upon reply and an additional lottery ticket upon reply within 1 week. The overall response rate across the three groups was 65.3%. The response rate was highest in the no incentive group (69.3%) and lowest in the one plus one lottery ticket group (62.0%). In a survival analysis, the difference between the two response curves was significant by the log-rank test (P = 0.04), with the no incentive group having a shorter time to response than the incentive group. Our findings suggest that the use of lottery tickets as incentives to increase participation in a mail questionnaire among parents may be less valuable or even harmful. Incentives may undermine motivation in studies in which the intrinsic motivation of the respondents is already high.     

Effectiveness of monetary incentives for recruiting adolescents to an intervention trial to reduce smoking

BACKGROUND: The study objective is to evaluate the effect of monetary incentives on response rates of adolescents to a smoking-related survey as the first step toward participation in an intervention trial. METHODS: A sample of 4,200 adolescent members of a managed care organization were randomized to one of four incentive groups: a $2 cash group, a $15 cash group, a $200 prize drawing group, or a no-incentive group. We compared group-specific response rates and willingness to be contacted about future study activities, as well as costs. RESULTS: Incentives increased survey response rates (55% response without incentive vs. a 69% response with incentive), with response of 74% in the $15 cash group, 69% in the token group, and 63% with a prize incentive. Incentives did not adversely affect willingness of adolescents to be contacted about a smoking intervention, (65% willing with incentives vs. 60% without, P = 0.03). In terms of cost per additional survey completed, token and prize groups were marginally more expensive than the no-incentive group ($0.40 and $1.42, respectively) while the large cash incentive was substantially more costly ($11.37). CONCLUSIONS: Monetary incentives improve response rates to a mailed survey, without adverse impact on willingness to further participate in intervention activities. However, a variety of issues must be considered when using incentives for recruitment to intervention studies.     

The effect of two mailing strategies on the response to a survey of physicians

In 1989, the authors tested the effectiveness of two response-enhancing techniques, a postage stamped or franked return envelope and a prenotification letter, in a survey of pregnancy among 10,047 resident physicians in the United States. The techniques were randomly assigned using a factorial design. No significant interactions were observed between the techniques. After two mailings, those who received a stamped return envelope had a response of 71.2%, compared with 68.2% for those who received a franked return envelope (95% confidence interval 1.3-4.9%). Men who received the stamped envelope had a 5.9% greater response than those who received the franked envelope (p less than 0.001), but the type of postage did not influence response among women (p = 0.84); this interaction was statistically significant (p = 0.006). Physicians who received a prenotification letter had a response of 69.0%, compared with 70.5% for those who did not receive the letter (95% confidence interval -3.3 to 0.2%). The authors conclude that seemingly minor changes in survey design could have saved from 12% to 19% of the total cost of the study.     

The use and influence of employee incentives on participation and throughput in a telephonic disease management program

Participation rates are often viewed by vendors and employer-based disease management (DM) services as an important benchmark of successful program implementation. Although participation is commonly understood to vary widely between and within employer groups, little is known about the role of incentives on rates of participation and graduation from DM programs. This study examined the use of incentives, employer characteristics, and perceptions of employee-employer communication on participation and program throughput. The relationship between incentive use and rates of participation and throughput among 87 employer groups from the 2004 company portfolio were assessed using existing account information. Detailed information on the highest and lowest third of the sample was obtained through interviews with account representatives. Wilcoxon, chi square, and regression analyses were used to examine the influence of employer characteristics and incentive factors on enrollee participation rates and program completion. Fifty-two percent of the accounts offered incentives for participation. From 1% to 23% of the eligible employees enrolled and completed the DM program. Incentives had a direct impact on participation, with amounts greater than $50 the most effective. Participation increased with communication tools including e-mail, high-blast (repeated) communications, and health fairs. Results suggest that cash incentives and communication play a significant role in rates of participation and program completion.     

A randomised controlled trial to determine the effect on response of including a lottery incentive in health surveys [ISRCTN32203485]

Background

Postal questionnaires are an economical and simple method of data collection for research purposes but are subject to non-response bias. Several studies have explored the effect of monetary and non-monetary incentives on response. Recent meta-analyses conclude that financial incentives are an effective way of increasing response rates. However, large surveys rarely have the resources to reward individual participants. Three previous papers report on the effectiveness of lottery incentives with contradictory results. This study aimed to determine the effect of including a lottery-style incentive on response rates to a postal health survey.

Methods

Randomised controlled trial. Setting: North and West Birmingham. 8,645 patients aged 18 or over randomly selected from registers of eight general practices (family physician practices). Intervention: Inclusion of a flyer and letter with a health questionnaire informing patients that returned questionnaires would be entered into a lottery-style draw for £100 of gift vouchers. Control: Health questionnaire accompanied only by standard letter of explanation. Main outcome measures: Response rate and completion rate to questionnaire.

Results

5,209 individuals responded with identical rates in both groups (62.1%). Practice, patient age, sex and Townsend score (a postcode based deprivation measure) were identified as predictive of response, with higher response related to older age, being female and living in an area with a lower Townsend score (less deprived).

Conclusion

This RCT, using a large community based sample, found that the offer of entry into a lottery style draw for £100 of High Street vouchers has no effect on response rates to a postal health questionnaire.     

Increasing response rates in physicians'' mail surveys: an experimental study.

It is becoming increasingly difficult to obtain high response rates in physicians' mail surveys. In 1983-84, we tested the effectiveness of two techniques among 604 Quebec physicians who had not responded to an initial letter. A handwritten thank you note at the bottom of the letter accompanying the questionnaire and a more personalized mailout package increased response rates by 40.7 per cent and 53.1 per cent, respectively, compared to control groups.     

A randomized trial of the impact of survey design characteristics on response rates among nursing home providers

An experiment was conducted to maximize participation of both the Director of Nursing (DoN) and the Administrator (ADMIN) in long-term care facilities. Providers in each of the 224 randomly selected facilities were randomly assigned to 1 of 16 conditions based on the combination of data collection mode (web vs. mail), questionnaire length (short vs. long), and incentive structure. Incentive structures were determined by amount compensated if the individual completed and an additional amount per individual if the pair completed (a) $30 individual/$5 pair/$35 total; (b) $10 individual/$25 pair/$35 total; (c) $30 individual/$20 pair/$50 total; and (d) $10 individual/$40 pair/$50 total. Overall, 47.4% of eligible respondents participated; both respondents participated in 29.3% of facilities. In multivariable analyses, there were no differences in the likelihood of both respondents participating by mode, questionnaire length, or incentive structure. Providing incentives contingent on participation by both providers of a facility was an ineffective strategy for significantly increasing response rates.     

Increasing response rates for mailed surveys of Medicaid clients and other low-income populations.

Mailing surveys to low-income populations is often avoided because of concern about low response rates. In this study, the authors used a mailed survey of a low-income population to test whether $1.00 or $2.00 cash-response incentives were worth the expense and whether 2-day priority mail ($2.90 postage) would yield a sufficiently higher response rate than certified mail ($1.52 postage) to justify its cost. In 1994, 2,243 randomly selected families in subsidized health care programs in Pierce County, Washington, were randomly sent no incentive, $1.00, or $2.00 in the first of three mailings. For the third mailing, nonrespondents were randomly assigned to receive either certified or 2-day priority mail. After 4 weeks, the response rates were 36.7%, 48.1%, and 50.3% for the no-incentive, $1.00, and $2.00 groups, respectively. After three mailings, the cost per response was the lowest for the group that received $1.00. The response rate for the certified mailing (28.1%) was significantly higher than the rate for the more expensive priority mailing (21.7%). No incentive-related bias was detected. The authors concluded that the most efficient protocol for this low-income population was to use a $1.00 incentive in the first mailing and a certified third mailing.     

Combining conditional and unconditional recruitment incentives could facilitate telephone tracing in surveys of postpartum women

BACKGROUND AND OBJECTIVE: To compare tracing and contact rates using alternative incentives in a computer-assisted telephone interview (CATI) survey among postpartum women. METHODS: In a randomized trial of 1,061 postpartum women 18-49 years of age selected from four Iowa counties, we compared the effects of: (1) unconditional $5 telephone card incentive enclosed with the introductory letter followed by $25 incentive conditional upon successful telephone tracing, contact, and completion of CATI (Group 1, n = 530) vs. (2) $30 incentive conditional upon subject completion of CATI (Group 2, n = 531). RESULTS: Overall telephone tracing and contact rates achieved were 67.8% and 66.6%, respectively. Tracing (70.2 vs. 65.4%, P = .09) and contact (68.5 vs. 64.8%, P = .26) rates were consistently higher among subjects assigned the combination of a conditional and an unconditional incentive. The combined incentive type had a greater impact on telephone tracing success rates for subjects on whom we could not initially locate an active telephone number (16.7 vs. 7.3%, P = .07) when compared to subjects for whom we found an active telephone number at the time of mailing the introductory letter (78.9 vs. 75.9%, P = .30). CONCLUSIONS: Combining conditional and unconditional recruitment incentives can facilitate telephone tracing efforts in surveys conducted among recently postpartum women.     

Can financial incentives and other nudges increase COVID-19 vaccinations among the vaccine hesitant? A randomized trial

Despite rapid initial uptake, COVID-19 vaccinations in the United States stalled within a few months of widespread rollout in 2021. In response, many state and local governments, employers and health systems used public health messaging, financial incentives and creative scheduling tools to increase vaccine uptake. Although these approaches drew on evidence from influenza and other vaccination efforts, they were largely untested in the context of SARS-CoV-2. In mid-2021, months after vaccines were widely available, we evaluated vaccination intentions and vaccine uptake using a randomized control trial. To do this, we recruited unvaccinated members of a Medicaid managed care plan in California (n = 2,701) and randomly assigned them to different public health messages, $10 or $50 financial incentives for vaccination, a simple vaccination appointment scheduler, or control. While messages increased vaccination intentions, none of the interventions increased vaccination rates. Estimates for financial incentives rule out even relatively small increases in vaccination rates. Small financial incentives and other behavioral nudges do not meaningfully increase COVID-19 vaccination rates amongst the vaccine hesitant.     

Physician response to surveys. A review of the literature

OBJECTIVE: Physician-specific surveys are a frequently used tool in health services research, but attempts at ensuring adequate response rates are rarely reported. We reviewed literature of survey methodology specific to physician surveys and report those found to be most effective. DATA SOURCES: Studies were identified by searching MEDLINE and PSYCHInfo from 1967 through February 1999. We included all English-language studies that randomized physician survey respondents to an experimental or control group. The authors independently extracted data from 24 studies examining survey methodology of physician-specific surveys. We included Mantel-Haenszel chi-squares comparing treatment groups, if present. If not, these were calculated from study data. RESULTS: Pre-notification of survey recipients, personalizing the survey mailout package, and nonmonetary incentives were not associated with increased response rates. Monetary incentives, the use of stamps on both outgoing and return envelopes, and short questionnaires did increase response rates. Few differences were reported in response rates of phone surveys compared with mail surveys and between the demographics and practice characteristics of early survey respondents and late respondents. CONCLUSIONS: We report some simple approaches that may significantly increase response rates of mail surveys. Surprisingly, the response rates of mail surveys of physicians compared favorably with those from telephone and personal interview surveys. Nonresponse bias may be of less concern in physician surveys than in surveys of the general public. Future research steps include specifically testing the more compelling results to allow for better control of confounders.     

Effect of Payment Incentives on Cancer Screening in Ontario Primary Care

PURPOSE

There is limited evidence for the effectiveness of pay for performance despite its widespread use. We assessed whether the introduction of a pay-for-performance scheme for primary care physicians in Ontario, Canada, was associated with increased cancer screening rates and determined the amounts paid to physicians as part of the program.

METHODS

We performed a longitudinal analysis using administrative data to determine cancer screening rates and incentive costs in each fiscal year from 1999/2000 to 2009/2010. We used a segmented linear regression analysis to assess whether there was a step change or change in screening rate trends after incentives were introduced in 2006/2007. We included all Ontarians eligible for cervical, breast, and colorectal cancer screening.

RESULTS

We found no significant step change in the screening rate for any of the 3 cancers the year after incentives were introduced. Colon cancer screening was increasing at a rate of 3.0% (95% CI, 2.3% to 3.7%) per year before the incentives were introduced and 4.7% (95% CI, 3.7% to 5.7%) per year after. The cervical and breast cancer screening rates did not change significantly from year to year before or after the incentives were introduced. Between 2006/2007 and 2009/2010, $28.3 million, $31.3 million, and $50.0 million were spent on financial incentives for cervical, breast, and colorectal cancer screening, respectively.

CONCLUSIONS

The pay-for-performance scheme was associated with little or no improvement in screening rates despite substantial expenditure. Policy makers should consider other strategies for improving rates of cancer screening.     

How to obtain excellent response rates when surveying physicians

This paper outlines ways to maximize response rates to surveysby summarizing the most relevant literature to date and demonstratinghow these techniques have resulted in consistently high ratesof return in family practice research. We describe the methodologyused in recent surveys of physicians conducted by the Centrefor Studies in Family Medicine through its Thames Valley FamilyPractice Research Unit, located in London, Ontario, Canada andfunded by the Ontario Ministry of Health and Long-Term Care.The identification and implementation of these techniques tomaximize response rates is critical, as primary health careresearchers often rely on information gathered through questionnairesto study physicians' practice profiles, experiences and attitudes.Four separate and distinct mailed surveys of physicians usinga modified Dillman approach were conducted from 2001 to 2004.The sampling strategies, topics, types of questions and responseformats of these surveys varied. The first survey did not useany incentives or recorded delivery/registered mail and receiveda response rate of 48%. In sharp contrast, the other three surveysobtained responses rates of 76%, 74%, 74%, respectively, achievedthrough the use of gift certificates and recorded delivery/registeredmail. Sending a survey by recorded delivery/registered mailtends to result in the survey package being given priority inthe physicians' incoming mail at the practice. Gift certificatespartially compensate physicians for time spent completing thesurvey and recognition of the time required is appreciated.The response rates achieved provide strong evidence to supportthe use of monetary incentives and recorded delivery/registeredmail (along with the Dillman approach) in survey research. Itis anticipated that this evidence will be used by other researchersto justify requests for funding to cover the costs associatedwith incentives and recorded delivery/registered mail. We recommendthe use of these strategies to maximize response rates and improvethe quality of this type of primary health care research. Keywords. Response rates, surveys, physicians.     

Lottery-based versus fixed incentives to increase clinicians' response to surveys

Objective. To compare the effects of lottery‐based and fixed incentives on clinicians' response to surveys. Data Sources. Three randomized trials with fixed payments and actuarially equivalent lotteries. Study Design. Trial 1 compared a low‐probability/high‐payout lottery, a high‐probability/low‐payout lottery, and no incentive. Trial 2 compared a moderate‐probability/moderate‐payout lottery with an unconditional fixed payment (payment sent with questionnaire). Trial 3 compared a moderate‐probability/moderate‐payout lottery with a conditional fixed payment (payment promised following response). Principal Findings. Neither the low‐probability nor high‐probability lotteries improved response compared with no incentive. Unconditional fixed payments produced significantly greater response than actuarially equivalent lotteries, but conditional fixed payments did not. Conclusions. Lottery‐based incentives do not improve clinicians' response rates compared with no incentives, and they are inferior to unconditional fixed payments.