Impact of a postcard versus a questionnaire as a first reminder in a postal lifestyle survey.

STUDY OBJECTIVE--The study aimed to consider the impact of two different types of reminder on response rates and costs in a postal survey. DESIGN--The study was a cross sectional survey. A self-completion lifestyle questionnaire was used. Those who did not respond after the initial mailing were randomly allocated to receive either a postcard or questionnaire as a first reminder. All outstanding non-responders received a questionnaire as a second reminder. SUBJECTS--A representative sample of 698 adults aged 16-70 was used, drawn from a family health services authority register. MAIN RESULTS--Postcard reminders were as effective as questionnaire reminders in increasing response whether one or two reminders are sent. The costs per response were calculated. Two questionnaires as reminders were found to be 1.7 times more expensive than a postcard plus questionnaire. Including the initial mailing, the cost per response using all questionnaires was 1.3 times the cost when a postcard was used for the first reminder. CONCLUSIONS--To increase the response to a postal survey effectively and economically, two reminders should be sent--first a postcard and then a questionnaire.     

Do incentives, reminders or reduced burden improve healthcare professional response rates in postal questionnaires? two randomised controlled trials

ABSTRACT: BACKGROUND: Healthcare professional response rates to postal questionnaires are declining and this may threaten the validity and generalisability of their findings. Methods to improve response rates do incur costs (resources) and increase the cost of research projects. The aim of these randomised controlled trials (RCTs) was to assess whether 1) incentives, 2) type of reminder and/or 3) reduced response burden improve response rates; and to assess the cost implications of such additional effective interventions. METHODS: Two RCTs were conducted. In RCT A general dental practitioners (dentists) in Scotland were randomised to receive either an incentive; an abridged questionnaire or a full length questionnaire. In RCT B non-responders to a postal questionnaire sent to general medical practitioners (GPs) in the UK were firstly randomised to receive a second full length questionnaire as a reminder or a postcard reminder. Continued non-responders from RCT B were then randomised within their first randomisation to receive a third full length or an abridged questionnaire reminder. The cost-effectiveness of interventions that effectively increased response rates was assessed as a secondary outcome. RESULTS: There was no evidence that an incentive (52% versus 43%, Risk Difference (RD) -8.8 (95%CI [MINUS SIGN]22.5, 4.8); or abridged questionnaire (46% versus 43%, RD [MINUS SIGN]2.9 (95%CI [MINUS SIGN]16.5, 10.7); statistically significantly improved dentist response rates compared to a full length questionnaire in RCT A. In RCT B there was no evidence that a full questionnaire reminder statistically significantly improved response rates compared to a postcard reminder (10.4% versus 7.3%, RD 3 (95%CI [MINUS SIGN]0.1, 6.8). At a second reminder stage, GPs sent the abridged questionnaire responded more often (14.8% versus 7.2%, RD [MINUS SIGN]7.7 (95%CI [MINUS SIGN]12.8, -2.6). GPs who received a postcard reminder followed by an abridged questionnaire were most likely to respond (19.8% versus 6.3%, RD 8.1%, and 9.1% for full/postcard/full, three full or full/full/abridged questionnaire respectively). An abridged questionnaire containing fewer questions following a postcard reminder was the only cost-effective strategy for increasing the response rate ([POUND SIGN]15.99 per response). CONCLUSIONS: When expecting or facing a low response rate to postal questionnaires, researchers should carefully identify the most efficient way to boost their response rate. In these studies, an abridged questionnaire containing fewer questions following a postcard reminder was the only cost-effective strategy. An increase in response rates may be explained by a combination of the number and type of contacts. Increasing the sampling frame may be more cost-effective than interventions to prompt non-responders. However, this may not strengthen the validity and generalisability of the survey findings and affect the representativeness of the sample.     

Prenotification but not envelope teaser increased response rates in a bulimia nervosa mental health literacy survey: A randomized controlled trial

ObjectivesUnderstanding attitudes to mental health issues can inform public health interventions. However, low response rates may contribute to nonresponse bias. In a randomized controlled trial we examined the effect of sending a prenotification postcard before the questionnaire and the placement of a short message on the survey envelope (teaser) on response rates to a mailed questionnaire about bulimia nervosa “mental health literacy”.Study Design and SettingQuestionnaires were mailed to 3,010 adults (50.6% female and 49.4% male) aged 18–65 years. In a 2 (pre-notification–present; absent) by 2 (teaser–present; absent) design, questionnaire recipients were randomly allocated to the experimental strategies. Outcomes considered were response rate, response time, and cost.ResultsThe overall response rate was 22.0%. Significant main effects showed higher response rates for the use of prenotification (present = 23.6%; absent = 20.3%), among female participants, and older participants. A significant interaction of teaser by gender indicated lower response rates for men who received the teaser but not for women. Older participants returned the questionnaire more promptly than younger participants. Females—but not males—who received the teaser were slower to return the questionnaire. Higher response rates for participants receiving the postcard compensated for increased costs, particularly for males and older participants.ConclusionResponse rates to a mental health postal survey can be increased through the use of prenotification.     

Response rates of Victorian general practitioners to a mailed survey on miscarriage: randomised trial of a prize and two forms of introduction to the research

AIMS: To identify whether the form of introduction to a study and knowledge of a substantial prize influence the response rate of general practitioners (GPs) to a postal survey. METHODS: A postal survey of 700 randomly selected Victorian GPs concerning management of early pregnancy bleeding and miscarriage, incorporating two randomised-controlled trials of recruitment methods; analysis of response rates and costs at 4 weeks and 11 weeks. RESULTS: The response rate was 61.5% of eligible participants. Doctors made aware of a prize were more likely to respond in the first four weeks (difference in response rate 10.2%, 95% confidence interval (CI) 2.8%-17.6%). This difference diminished after the first four weeks. Doctors introduced to the survey by a telephone call were no more likely to respond than those introduced by a postcard. The use of a postcard saved 73% of the cost of introducing the survey by telephone. Female doctors were more likely than males to reply (difference 12.3%, 95% CI 4.7%-19.9%). Rural doctors were no more likely to reply than urban doctors. Very few doctors (16.2%) completed a Practice Assessment activity associated with the survey. CONCLUSIONS: A valuable prize will accelerate response to a survey by GPs, thereby reducing the costs of follow-up. The cost of telephoned introductions is not justified, when compared with a brief written introduction.     

Introductory postcards: do they increase response rate in a telephone survey of older persons?

The purpose of this study was to assess whether contact by mail via an introductory postcard would increase the response rate to a telephone survey in a selected population of 348 people 60 years and older. Of this group, 208 were sent a postcard introducing the study and informing them that someone would call in the next 1-2 weeks to conduct a telephone interview. The remaining sample of 140 did not receive the introductory postcard. Amongst the potential participants who were contacted and able to participate (n = 240), the response rate for completed interview was 58.0% for those sent a postcard and 42.3% for those who were not. The refusal rate was 25.3% (postcard sent) and 37.2% (not sent a postcard). When the outcome of contact is collapsed to two categories, 'agreed to participate' and 'refusal', the response rate to participate was 73.5% (sent a postcard) and 59.0% (not sent a postcard) (chi(2) 5.14, d.f. 1, sign. 0.023). The response rate amongst those who recall being sent a postcard rose to 86.0%. Whether agreeing to participate or refusing to be involved in the survey was dependent on being sent an introductory postcard. The conclusion is that mailing a postcard prior to the first telephone contact increases the participation rates of older persons in a telephone survey at very little extra cost (A$0.60 per person).     

Cost-effectiveness of a lottery for increasing physicians' responses to a mail survey

To evaluate the cost-effectiveness of a lottery on physicians' responses to a mail survey, a randomized controlled trial was conducted with a random sample of 1,000 members of the Quebec Federation of General Practitioners in 1997. For the first mailing of this survey, each respondent was randomly assigned to the control or experimental group, which was offered participation in a lottery upon return of the questionnaire. Response rate was 41.2% in the experimental group and 34.8% in the control group, a 6.4% difference (CI95%: 0.6%-12.6%). The additional cost of the lottery was about Can$500, giving an incremental cost of Can$16 per questionnaire returned. In conclusion, a lottery resulted in a small but statistically significant increase in the response rate of physicians to a mail survey. This method may be a cost-effective option when applied to large surveys.     

Assessing delivery practices of mothers over time and over space in Uganda, 2003–2012

The issue of poor response rates to population surveys has existed for some decades, but few studies have explored methods to improve the response rate in follow-up population cohort studies. A sample of 100,000 adults from the 45 and Up Study, a large population cohort in Australia, were followed up 3.5 years after the baseline cohort was assembled. A pilot mail-out of 5000 surveys produced a response rate of only 41.7 %. This study tested methods of enhancing response rate, with three groups of 1000 each allocated to (1) receiving an advance notice postcard followed by a questionnaire, (2) receiving a questionnaire and then follow-up reminder letter, and (3) both these strategies. The enhanced strategies all produced an improved response rate compared to the pilot, with a resulting mean response rate of 53.7 %. Highest response was found when both the postcard and questionnaire reminder were used (56.4 %) but this was only significantly higher when compared to postcard alone (50.5 %) but not reminder alone (54.1 %). The combined approach was used for recruitment among the remaining 92,000 participants, with a resultant further increased response rate of 61.6 %. Survey prompting with a postcard and a reminder follow-up questionnaire, applied separately or combined can enhance follow-up rates in large scale survey-based epidemiological studies.     

A mail survey of United States hematologists and oncologists: a comparison of business reply versus stamped return envelopes

Mailed surveys are a popular means of obtaining data on large populations. In July 1999 a mail survey was conducted among 3000 randomly selected members of the American Society of Hematology to assess their approach to diagnosis and treatment of polycythemia vera. Because the researchers and the study population are members of the same professional organization with a vested interest in the results, we anticipated that the advantages of return stamped postage seen in previous studies would be less significant. The response rate for stamped return envelopes was 38% versus 32% for business reply envelopes. This statistically significant difference (P =.0005) of six percentage points is comparable to previous research. Excluding labor, the total cost per returned survey was $2.62 for business reply envelopes versus $1.82 for stamped return envelopes. We conclude that stamped return envelopes are a more effective and cost-efficient means of procuring data from physician specialists.     

Surveying nurses: identifying strategies to improve participation

While surveys of nurses are an important tool in health services and policy research, they are often characterized by low response rates. The authors conducted a systematic review of 22 published reports of efforts to improve response rates to nurse surveys. Two general strategies were explored in this literature: incentive and design-based approaches. Even small financial incentives were found to be effective in improving nurse survey participation. Token nonmonetary incentives, in contrast, were much less effective. In terms of design strategies, postal and telephone strategies have generally been more successful than have fax or web-based approaches, with evidence also supporting use of mixed-mode surveys in this population. In addition, use of first-class stamps on return envelopes as well as questionnaires personalized and endorsed by legitimizing professional associations were also more likely to be successful. Researchers should continue to implement evidence-based strategies in order to improve the survey response of nurses.     

An Analysis of Response Rate and Economic Costs Between Mail and Web-Based Surveys Among Practicing Dentists: A Randomized Trial

This study explored the economic costs and response rate of mail and web-based surveys with practicing dentists. A random sample of 6,000 practicing dentists was randomly assigned into three groups of 2,000: choice (mail or web-based), postal mail, or web-based. The Florida Tobacco Control Survey 2009, which is composed of 28 questions (including subject demographic questions), served as the survey instrument. A total of 1,232 surveys were returned by the three different groups (21% overall response rate). Response rates were best for the mail (26%) with the worst response rate coming from the Web group (11%). However, a cost-effectiveness analysis revealed that web surveys were 2.68 times more cost effective.     

Effectiveness of monetary incentives for recruiting adolescents to an intervention trial to reduce smoking

BACKGROUND: The study objective is to evaluate the effect of monetary incentives on response rates of adolescents to a smoking-related survey as the first step toward participation in an intervention trial. METHODS: A sample of 4,200 adolescent members of a managed care organization were randomized to one of four incentive groups: a $2 cash group, a $15 cash group, a $200 prize drawing group, or a no-incentive group. We compared group-specific response rates and willingness to be contacted about future study activities, as well as costs. RESULTS: Incentives increased survey response rates (55% response without incentive vs. a 69% response with incentive), with response of 74% in the $15 cash group, 69% in the token group, and 63% with a prize incentive. Incentives did not adversely affect willingness of adolescents to be contacted about a smoking intervention, (65% willing with incentives vs. 60% without, P = 0.03). In terms of cost per additional survey completed, token and prize groups were marginally more expensive than the no-incentive group ($0.40 and $1.42, respectively) while the large cash incentive was substantially more costly ($11.37). CONCLUSIONS: Monetary incentives improve response rates to a mailed survey, without adverse impact on willingness to further participate in intervention activities. However, a variety of issues must be considered when using incentives for recruitment to intervention studies.     

Methodologies for improving response rates in surveys of physicians: a systematic review

Although physician surveys are an important tool in health services and policy research, they are often characterized by low response rates. The authors conducted a systematic review of 66 published reports of efforts to improve response rates to physician surveys. Two general strategies were explored in this literature: incentive and design-based approaches. Even small financial incentives were found to be effective in improving physician response. Token nonmonetary incentives were much less effective. In terms of design strategies, postal and telephone strategies have generally been more successful than have fax or Web-based approaches, with evidence also supporting use of mixed-mode surveys in this population. In addition, use of first-class stamps on return envelopes and questionnaires designed to be brief, personalized, and endorsed by legitimizing professional associations were also more likely to be successful. Researchers should continue to implement design strategies that have been documented to improve the survey response of physicians.     

Estimation of outpatient risk characteristics and influenza vaccination status: validation of a self-administered questionnaire.

As part of an effort to evaluate the effectiveness of a multifaceted, institution-wide influenza vaccination program, we mailed a self-administered postcard questionnaire to 500 randomly selected outpatients following both the 1987-1988 and 1988-1989 immunization seasons. The postcard asked if the patients received influenza vaccine, where they were immunized, whether they had any high-risk medical illnesses, their age at their last birthday, and their name. To assess the validity of the questionnaire, following the 1988 survey, we selected a 25% systematic sample of respondents and compared survey responses with medical record documentation. Response rates for the surveys were 77.6% for the 1988 survey (two mailings) and 86.5% for the 1989 survey (three mailings). Ninety-four percent (n = 89) of charts selected for audit were successfully reviewed. Agreement between questionnaire responses and medical records was excellent, with greater than 90% agreement and kappas of .8 to .9 for classifying patients as high-risk and receipt of vaccine at the Department of Veterans' Affairs Medical Center. For other data, agreement ranged from 84% to 96% with kappas of .6 to .75. Responses to the 1989 survey also demonstrated the importance of sample frame definition and response wave on estimates of risk characteristics and vaccination rates. A self-administered postcard questionnaire is a valid tool for estimating outpatient risk characteristics and influenza vaccination rates.     

A comparison of general practitioner response rates to electronic and postal surveys in the setting of the National STI Prevention Program

Objective: To compare the response rates achieved for an online survey with a postal survey of general practitioners (GPs) as a method to evaluate the National STI Prevention Program. Methods: All GPs in Australia were asked to complete an online survey. A further sample of 509 GPs were asked to complete a postal survey. Response rates to both recruitment methods were compared. The demographic characteristics of responders were compared to the entire GP population of Australia. Results: Twenty GPs completed the online survey (response rate <0.1%). Sixty‐three GPs completed the postal survey (response rate 12.4%). The demographic characteristics of those responding to the postal survey showed no statistically significant difference compared to the general GP population. Conclusion: Our postal survey had a higher response rate than the online survey. Our response to the postal survey was lower than other similar studies and is likely to be due to a lack of incentives and follow‐up. Even with the low response rate it appears that postal surveys can provide a good representation of the overall population. Implications: Despite growing use of online surveys, postal surveys should still be the method of choice whenever possible. Postal surveys should include incentives and further follow‐up of the initial recruitment should be conducted.     

Telephone reminders are a cost effective way to improve responses in postal health surveys

STUDY OBJECTIVE: To assess the effectiveness of a telephone reminder in increasing responses to postal surveys and to calculate the differential costs per completed questionnaire. DESIGN: Randomised controlled trial. SETTING: Australian university and rehabilitation medicine practice. PARTICIPANTS: The trial was conducted in 1999 among the 143 non-respondents to a questionnaire about work related neck and upper body disorders. The questionnaire was sent to two Australian female samples: 200 office workers (Sample A) and 92 former rehabilitation medicine patients (Sample B). A reminder letter, another copy of the questionnaire and a final letter were sent at two week intervals. Half of the non-respondents within each sample were randomly selected to receive a telephone reminder just after the second mailout of the questionnaire. All direct costs were calculated. MAIN RESULTS: Responses were significantly higher among those who received the telephone reminder intervention (relative risk 2.54, 95% confidence intervals 1.43 to 4.52). Analysed by intention to phone, 47% of non-respondents in Sample A and 38% in Sample B returned a complete questionnaire after the intervention, compared with 21% and 10%, respectively, in the control groups. For the 112 women (combined samples) who returned completed questionnaires before randomisation, the average cost per respondent was AUD14. There was a higher total cost for the intervention groups (AUD851 versus AUD386 for controls), but the significantly higher number of additional completed responses (31 versus 12) resulted in a 15% lower marginal cost per completed questionnaire in those groups. CONCLUSION: Telephone reminders are cost effective in improving responses to postal surveys.     

Applying Mail Response Enhancement Techniques to Health Care Surveys

A study of over 1200 persons gave insights into the effectiveness of different appeals in a mail survey for a health-care facility. The influences of financial incentives, contributions to charity and time appeals were studied. Not only was the practical effectiveness of the three different appeals investigated, the cost of obtaining the response rates with each was also shown. The range of response rates was from 25% to 67%. The average cost per return for the different methods ranged from $2.87 to $5.64. A method of testing for the potential bias in a survey is also presented.     

Applying mail response enhancement techniques to health care surveys: a cost-benefit approach

A study of over 1200 persons gave insights into the effectiveness of different appeals in a mail survey for a health-care facility. The influences of financial incentives, contributions to charity and time appeals were studied. Not only was the practical effectiveness of the three different appeals investigated, the cost of obtaining the response rates with each was also shown. The range of response rates was from 25% to 67%. The average cost per return for the different methods ranged from $2.87 to $5.64. A method of testing for the potential bias in a survey is also presented.     

Cost-effectiveness of follow-up contact for a postal survey: a randomised controlled trial

Objective: This study examines the effectiveness and costs of follow‐up phone calls in improving response rates to a community survey. Methods: Non‐responders to a postal survey were randomly allocated to receive a phone call or no phone call. The resources used for the development and implementation of the survey were documented. The response rates and cost per level of follow‐up contact examined. Results: Follow‐up phone calls led to a statistical significant increase in the number of responses to a community‐wide survey, relative to no phone call. This relative increase in responses (n=62 for the follow‐up phone call group versus n=1 for controls), did not increase the absolute survey response rate sufficiently (from 38.5% for two mailed surveys to 39.8% for two mailed surveys plus a phone call) to justify the phone call costs. Scenario analyses show increasing the initial response rate by 10% and conducting a second mailed survey achieves greater marginal cost savings than increasing the response rate to the second mailout or the follow‐up phone calls. Conclusions: These results suggest a follow‐up phone call was not cost effective. Survey research ought to primarily focus on obtaining optimal initial response rates by using strategies identified in a Cochrane meta‐analytic review.     

Physician response to surveys. A review of the literature

OBJECTIVE: Physician-specific surveys are a frequently used tool in health services research, but attempts at ensuring adequate response rates are rarely reported. We reviewed literature of survey methodology specific to physician surveys and report those found to be most effective. DATA SOURCES: Studies were identified by searching MEDLINE and PSYCHInfo from 1967 through February 1999. We included all English-language studies that randomized physician survey respondents to an experimental or control group. The authors independently extracted data from 24 studies examining survey methodology of physician-specific surveys. We included Mantel-Haenszel chi-squares comparing treatment groups, if present. If not, these were calculated from study data. RESULTS: Pre-notification of survey recipients, personalizing the survey mailout package, and nonmonetary incentives were not associated with increased response rates. Monetary incentives, the use of stamps on both outgoing and return envelopes, and short questionnaires did increase response rates. Few differences were reported in response rates of phone surveys compared with mail surveys and between the demographics and practice characteristics of early survey respondents and late respondents. CONCLUSIONS: We report some simple approaches that may significantly increase response rates of mail surveys. Surprisingly, the response rates of mail surveys of physicians compared favorably with those from telephone and personal interview surveys. Nonresponse bias may be of less concern in physician surveys than in surveys of the general public. Future research steps include specifically testing the more compelling results to allow for better control of confounders.     

Increasing response to a postal survey of sedentary patients - a randomised controlled trial [ISRCTN45665423]

Background

A systematic review identified a range of methods, which can influence response rates. However, analysis specific to a healthcare setting, and in particular, involving people expected to be poor responders, was missing, We examined the effect of pre-warning letters on response rates to a postal survey of sedentary patients whom we expected a low rate of response.

Methods

Participants were randomised to receive a pre-warning letter or no pre-warning letter, seven days before sending the main questionnaire. The main questionnaire included a covering letter and pre-paid return envelope. After seven days, non-responders were sent a reminder letter and seven days later, another reminder letter with a further copy of the questionnaire and return envelope.

Results

627 adults, with a mean age of 48 years (SD 13, range 18 to 78) of whom 69.2% (434/627) were women, were randomised. 49.0% (307/627) of patients were allocated to receive a pre-warning letter and 51.0% (320/627) no pre-warning letter, seven days in advance of posting the main questionnaire. The final response rate to the main questionnaire was 30.0% (92/307) amongst those sent a pre-warning letter and 20.9% (67/320) not sent a pre-warning letter, with an adjusted odds ratio of 1.60 (95% CI 1.1, 2.30).

Conclusions

The relatively low cost method of sending a pre-warning letter had a modest impact on increasing response rates to a postal questionnaire sent to a group of patients for whom a low response rate was anticipated. Investigators should consider incorporating this simple intervention when conducting postal surveys, to reduce the potential for nonresponse bias and to increase the study power. Methods other than postal surveys may be needed however when a low response rate to postal surveys is likely.