Questionnaire color and response rates to mailed surveys. A randomized trial and a meta-analysis

The authors conducted a randomized trial in Geneva, Switzerland, to assess whether response rates to a mailed survey could be increased by printing the questionnaire on green paper. The authors also conducted a meta-analysis of 10 experimental studies that tested the effect of colored questionnaires on response rates. The randomized trial showed no effect (relative risk of responding [RR] = 1.00). The meta-analysis showed that mailing questionnaires on pink paper increased response rates by 12% (RR = 1.12, 95% confidence interval = 1.01 to 1.25, p = 0.04). Other colors had no statistically significant effect (blue: RR = 1.03, p = 0.49; green: RR = 1.02, p = 0.23; yellow: RR = 0.96, p = 0.30). Overall, using colored instead of white paper had no effect (RR = 1.02, p = 0.17). Thus, printing questionnaires on colored paper does not substantially increase response rates in surveys, except for pink paper.     

The effects of continuing medical education credits on physician response rates to a mailed questionnaire

This study investigated whether the opportunity to obtain Continuing Medical Education (CME) credit together with a five-dollar bill increased response rates and questionnaire completion rates in a physician survey involving mailed questionnaires. One thousand, three hundred and fourteen cardiologists, family practitioners, general internists (non-surgeons) and 264 vascular surgeons randomly identified from the American Medical Association database participated. After two, of up to four, questionnaire mailings, the opportunity to obtain CME credit and a five-dollar bill were included with questionnaire mailings. Among non-surgeons, 26.5% responded to pre-incentive mailings and 30.2% of those initially unresponsive replied after the interventions. Among surgeons, 39% responded to pre-incentive mailings and 32.7% of those initially unresponsive replied after the interventions. In conclusion, the opportunity to receive CME credit combined with a small monetary incentive is an effective motivation for physicians participating in a study involving mailed questionnaires.     

Reported response rates to mailed physician questionnaires

OBJECTIVE: To examine response rate information from mailed physician questionnaires reported in published articles. DATA SOURCES/STUDY SETTING: Citations for articles published between 1985 and 1995 were obtained using a key word search of the Medline, PsychLit, and Sociofile databases. STUDY DESIGN: A 5 percent random sample of relevant citations was selected from each year. DATA COLLECTION/EXTRACTION METHODS: Citations found to be other than physician surveys were discarded and replaced with the next randomly assigned article. Selected articles were abstracted using a standardized variable list. PRINCIPAL FINDINGS: The average response rate for mailed physician questionnaires was 61 percent. The average response rate for large sample surveys (> 1,000 observations) was 52 percent. In addition, only 44 percent of the abstracted articles reported a discussion of response bias, and only 54 percent reported any type of follow-up. CONCLUSIONS: (1) Response rates have remained somewhat constant over time, and (2) researchers need to document the efforts used to increase response rates to mailed physician questionnaires.     

Ethnicity, but not cancer family history, is related to response to a population-based mailed questionnaire

PURPOSE: To determine if family history and ethnic background are factors affecting response to a mailed cancer family history questionnaire from the Ontario Familial Breast Cancer Registry. METHODS: Individuals diagnosed with primary invasive breast carcinomas (probands) were mailed a family history questionnaire, the first contact in a multi-stage process. This questionnaire obtained cancer family history and ethnicity data. After one month, a follow up telephone call was made to those who did not return this questionnaire and attempts were made to ask similar questions by telephone interview. Characteristics of those responding to the mailed questionnaire were compared to those who responded to the telephone interview only. RESULTS: 339 probands were included in this study: 242 returned a mailed version of the questionnaire; 57 completed the questionnaire over the phone. Cancer family history/genetic risk criteria was not significantly related to type of response. Probands identifying themselves as visible minorities were significantly less likely to respond to the mailed questionnaire than the telephone interview (11.6% vs. 22.8%, P=0.03). CONCLUSIONS: Having a family history of cancer did not appear to influence response to a mailed questionnaire, but those reporting an ethnic/racial background other than White were more likely to respond to a telephone interview.     

The effect on response rates of offering a small incentive with a mailed questionnaire.

To determine whether response rates to a mailed questionnaire sent to population control subjects could be increased through offer of a small incentive, half of the control subjects (n = 477) in a case-control study of renal cell carcinoma were randomly selected to receive a contact letter offering a lottery ticket if a completed questionnaire was returned; the remaining subjects (n = 477) received the same letter but with no mention of a lottery ticket. Overall response rates did not differ between the two groups (72.6% versus 74.4%), although a higher percentage of those offered a lottery ticket responded without follow-up (24.4% versus 18.5%). Binomial regression modeling of the effect of the lottery ticket offer, sex, age, and percent of urban dwellers on response indicated a significant effect only for percent of urban dwellers, the rate of response increasing with a decreasing percentage of urban dwellers. The effect of sex was of borderline significance (P = 0.05), with females having the higher rate of response.     

In a mailed physician survey, questionnaire length had a threshold effect on response rate

OBJECTIVE: To examine the association between questionnaire length and response rate in a mailed survey of generalist physicians randomly selected from the American Medical Association master file. STUDY DESIGN AND SETTING: In a pilot study, otherwise similar questionnaires of 30 different lengths (849 to 1,867 words) were mailed to 192 physicians in April 1999. In the main study, questionnaires of 16 different lengths (564 to 988 words) were mailed to 1,700 physicians between June 1999 and January 2000. RESULTS: In the pilot study, response rate decreased from 60% for questionnaires 849 words in length to 16.7% for questionnaires over 1,800 words in length. Logistic regression revealed an odds ratio of 0.887 (95%CI 0.813, 0.968; p=0.006) for word count, expressed in units of 100 words. In the main study, response rate varied between 51.5% and 71.4%. Logistic regression showed no association between response and word count (OR 0.988; 95%CI 0.896, 1.090; p=0.81). CONCLUSION: There appears to have been a threshold in these studies of approximately 1,000 words. Questionnaires above the threshold had lower response rates than those below it (38.0% vs. 59.4%).     

Telephone reminder calls increased response rates to mailed study consent forms

BACKGROUND: This study assessed the impact of follow-up reminder phone calls on response rates to a mailed consent form packet. METHODS: Patients with rheumatoid arthritis were invited to enroll in a study by signing and returning consent forms by mail. Patients not returning completed study consent forms were called and reminded to return the signed consent forms. RESULTS: Among 724 mailed consent form packets, 376 (52%) were returned without further follow-up. Follow-up reminder calls were made to 220 of the 348 patients who did not return signed consent forms. Among subjects contacted by phone, 67 (31% of those called) returned signed consent forms. CONCLUSION: Follow-up reminder phone calls raised the overall consent rate of 52 to 61%, suggesting that they can be an effective technique in increasing response rates.     

Randomized trial of financial incentives and delivery methods for improving response to a mailed questionnaire

In a follow-up study, only 64% of 126,628 US radiologic technologists completed a questionnaire during 1994-1997 after two mailings. The authors conducted a randomized trial of financial incentives and delivery methods to identify the least costly approach for increasing overall participation. They randomly selected nine samples of 300 nonresponders each to receive combinations of no, 1.00 US dollar, 2.00 US dollars, and 5.00 US dollars cash or check incentives delivered by first-class mail or Federal Express. Federal Express delivery did not achieve greater participation than first-class mail (23.2% vs. 23.7%). In analyses pooled across delivery methods, the response was significantly greater for the 2.00 US dollar bill (28.9%, 95% confidence interval (CI): 25.2, 32.7; p < 0.0001), 5.00 US dollars check (27.5%, 95% CI: 22.5, 33.0; p = 0.0001), 1.00 US dollar bill (24.6%, 95% CI: 21.2, 28.3; p = 0.0007), and 2.00 US dollars check (21.8%, 95% CI: 18.5, 25.3; p = 0.02) compared with no incentive (16.6%, 95% CI: 13.7, 19.9). The response increased significantly with increasing incentive amounts from 0.00 to 2.00 US dollars cash (p trend < 0.0001). The 2.00 US dollar bill achieved a 30% greater response than did a 2.00 US dollars check (p = 0.005). For incentives sent by first-class mail, the 5.00 US dollars check yielded 30% greater participation than did the 2.00 US dollars check (p = 0.07). A 1.00 US dollar bill, chosen instead of the 2.00 US dollars bill because of substantially lower overall cost and sent by first-class mail to the remaining 42,717 nonresponders, increased response from 64% to 72%.     

Print format and sender recognition were related to survey completion rate

OBJECTIVE: To assess whether print format (single-sided vs. double) and sender recognition (known vs. unknown) affect response and completion rates among physician survey respondents. STUDY DESIGN AND SETTING: Postal survey of 399 members of the Canadian Association of Emergency Physicians; 2 x 2 factorial design. RESULTS: Response rate was 69.4%. Single-sided printing yielded 7.4% (odds ratio OR = 1.41; 95% confidence interval CI = 0.90-2.20; P = 0.13), and a known sender yielded a 6.3% greater response rate (OR = 0.73; 95% CI = 0.47-1.14; P = 0.16). Overall item completion was 98.2%; items missed per respondent ranged from 1 to 14 out of 50. Print format and sender recognition interacted in predicting completion rate (OR = 13.33; 95% CI = 3.10-57.4; P = 0.001); completion was higher for double-sided printing with an unknown sender, and for single-sided printing with a known sender. Completion was also lower when response came after later mailouts (chi2(2) = 10.13; P = 0.006). CONCLUSIONS: Print format and sender recognition both yielded 6%-7% (nonsignificant) response rate differences. Survey completion rate varied even when overall item completion was high. Completion rate was useful for identifying subgroups likely to provide incomplete data (i.e., late responders), and may provide important information for subsequent surveys. Combining factors that on their own improve survey response may have unexpected consequences.     

Web-based and mailed questionnaires: a comparison of response rates and compliance

BACKGROUND: We assessed response rates and compliance for a printed questionnaire and a Web questionnaire in a Swedish population-based study and explored the influence of adding personalized feedback to the Web questionnaire. METHODS: We assigned 875 subjects to 1 of 3 groups: printed questionnaire, plain Web questionnaire, or Web questionnaire with personalized feedback. The questionnaire had 2 parts, first a general section and then a dietary section. RESULTS: The response rate for the general section was 64% for the printed questionnaire, compared with 50% for the Web questionnaire with feedback. For the dietary questionnaire, the rates were reversed, resulting in a total response rate for the dietary questionnaire that did not differ between printed and web questionnaire with feedback. CONCLUSIONS: Interactivity in the Web questionnaire increased compliance in completion of the second section of the questionnaire. Web questionnaires can be useful for research purposes in settings in which Internet access is high.     

Characteristics of respondents and non-respondents to a mailed questionnaire.

In establishing a cohort of U.S. nurses, an assessment of response bias was made comparing respondents and non-respondents with regard to age, education, state of residence, employment status, field of employment, and major specialty. Overall, the 122,328 respondents (69.7 per cent) and 43,222 non-respondents were quite similar. Together with the reasonable response rate in a homogeneous population, this suggests that estimation of exposure-disease associations is unlikely to be affected by major bias due to non-response.     

Increasing response rates for mailed surveys of Medicaid clients and other low-income populations.

Mailing surveys to low-income populations is often avoided because of concern about low response rates. In this study, the authors used a mailed survey of a low-income population to test whether $1.00 or $2.00 cash-response incentives were worth the expense and whether 2-day priority mail ($2.90 postage) would yield a sufficiently higher response rate than certified mail ($1.52 postage) to justify its cost. In 1994, 2,243 randomly selected families in subsidized health care programs in Pierce County, Washington, were randomly sent no incentive, $1.00, or $2.00 in the first of three mailings. For the third mailing, nonrespondents were randomly assigned to receive either certified or 2-day priority mail. After 4 weeks, the response rates were 36.7%, 48.1%, and 50.3% for the no-incentive, $1.00, and $2.00 groups, respectively. After three mailings, the cost per response was the lowest for the group that received $1.00. The response rate for the certified mailing (28.1%) was significantly higher than the rate for the more expensive priority mailing (21.7%). No incentive-related bias was detected. The authors concluded that the most efficient protocol for this low-income population was to use a $1.00 incentive in the first mailing and a certified third mailing.     

Questionnaire order significantly increased response to a postal survey sent to primary care physicians

OBJECTIVE: Primary care physicians are increasingly being asked to participate in postal surveys. Difficulties in achieving adequate response rates among physicians have been reported. We investigated the effect of two low-cost interventions on response to a primary care physician postal questionnaire. STUDY DESIGN AND SETTING: A 2x2 factorial trial was developed within the context of a national survey assessing views and practices of physicians regarding prostate-specific antigen testing. We evaluated questionnaire order (version 1: demographics first, version 2: topic-specific questions first) and written precontact. A national database of primary care physicians was compiled. One thousand five hundred ninety-nine physicians were randomly selected, stratified by health board, and randomized. RESULTS: 47.9% of eligible physicians completed a questionnaire. There was a statistically significant 5.1% higher response rate among physicians receiving version 1 of the questionnaire than those receiving version 2 (50.6% vs. 45.4%, P=0.05); the adjusted odds of response were significantly raised (odds ratio=1.24; 95% confidence interval=1.01-1.54). Precontact resulted in a nonsignificant 3.6% increase in response (49.8% vs. 46.2%; P=0.16). The interventions did not interact. CONCLUSION: Ordering questionnaires with general questions first can significantly increase response rates, whereas precontact can achieve a modest increase. These strategies may enhance response while adding little to the cost of a physician survey.     

Medical consumption of respondents and non-respondents to a mailed health survey

Non-response bias can distort the results of health surveys.The occurrence of selective non-response can be assessed whendata are available for both respondents and non-respondents.The objective of this study was to compare the medical consumptionof respondents and non-respondents to a mailed health survey.A mailed health survey was conducted among approximately 13,500adults and among parents of approximately 1,500 children aged5–15 years. The net response rate was 70.4%. A panel dataset that could be matched with the health survey data was availablefor all eligible persons. This data set comprises administrativeinformation on hospitalizations, annual health care expendituresand demographic variables. The results of this study show thatresponse was associated with age, sex, degree of urbanizationand type of insurance. After correcting for differences in demographicvariables, respondents and non-respondents differ in the utilizationof several types of care. Relatively more users than non-usersresponded. Response was not associated with the utilizationof care related to severe conditions such as in-patient hospitalcare. The conclusion from this study is that when a mailed healthsurvey is used to measure medical consumption, the non-responsebias will result in a small overestimation of utilization.     

Lottery incentives did not improve response rate to a mailed survey: a randomized controlled trial

OBJECTIVES: Our study aimed to examine the effect of an instant lottery ticket incentive on the response rate to a mailed questionnaire in a population of trauma patients. STUDY DESIGN AND SETTING: A randomized controlled trial at a major trauma center with 728 patients randomized into 2 groups prior to mailing of a questionnaire. One group of patients (Group A) had a sentence inserted into the cover letter stating that they would receive a $4 instant lottery ticket upon receipt of a completed questionnaire; Group B did not have an incentive. The response rate for both groups was measured after the initial mailing and at the end of the study. The results were analyzed using the X2 test to compare 2 proportions and a P value of .05 was considered significant. RESULTS: The early response rate in Group A was lower than in Group B, and the response rates for both groups were similar at final follow-up. The differences at both time periods were not statistically significant. CONCLUSION: The use of an instant lottery ticket incentive did not improve the response rate to a mailed questionnaire.