Including the questionnaire with an invitation letter did not improve a telephone survey's response rate

OBJECTIVE: To determine whether including the study questionnaire with a letter of invitation improves the response rate in a telephone-based survey. STUDY DESIGN AND SETTING: This randomized controlled trial was part of a larger study to assess patient preferences for novel and controversial treatments for inflammatory bowel disease at Royal Prince Alfred Hospital, a tertiary referral teaching hospital in Sydney, Australia. RESULTS: Of 270 eligible patients, 124 (46%) were randomized to receive the questionnaire plus invitation whereas 146 (54%) were in the control group receiving a letter of invitation only. The consent rate was 26% for those receiving the questionnaire and 36% for the control group. The odds ratio for consent to participate among those sent the questionnaire to those not sent the questionnaire was 0.63 (95% CI=0.37-1.07). CONCLUSION: This study found that the advance mailing of a questionnaire to potential participants in a telephone survey reduced the likelihood of their participation.     

Response rates of Victorian general practitioners to a mailed survey on miscarriage: randomised trial of a prize and two forms of introduction to the research

AIMS: To identify whether the form of introduction to a study and knowledge of a substantial prize influence the response rate of general practitioners (GPs) to a postal survey. METHODS: A postal survey of 700 randomly selected Victorian GPs concerning management of early pregnancy bleeding and miscarriage, incorporating two randomised-controlled trials of recruitment methods; analysis of response rates and costs at 4 weeks and 11 weeks. RESULTS: The response rate was 61.5% of eligible participants. Doctors made aware of a prize were more likely to respond in the first four weeks (difference in response rate 10.2%, 95% confidence interval (CI) 2.8%-17.6%). This difference diminished after the first four weeks. Doctors introduced to the survey by a telephone call were no more likely to respond than those introduced by a postcard. The use of a postcard saved 73% of the cost of introducing the survey by telephone. Female doctors were more likely than males to reply (difference 12.3%, 95% CI 4.7%-19.9%). Rural doctors were no more likely to reply than urban doctors. Very few doctors (16.2%) completed a Practice Assessment activity associated with the survey. CONCLUSIONS: A valuable prize will accelerate response to a survey by GPs, thereby reducing the costs of follow-up. The cost of telephoned introductions is not justified, when compared with a brief written introduction.     

Recruitment to mammography screening: a randomised trial and meta-analysis of invitation letters and telephone calls

OBJECTIVES: To determine the effectiveness of three recruitment strategies to encourage women to attend for an initial mammography screen, and to compare results with similar service studies. Interventions were: (1) an invitation letter; (2) two invitation letters; and (3) an invitation letter plus a follow-up telephone call. METHODS: All women aged 50-54 years in two BreastScreen New South Wales (BSNSW) Screening and Assessment Service catchment areas (n=3,144) were recruited from the Australian Electoral Roll and randomised to the four groups. Response rates for each intervention were compared relative to standard practice (one invitation letter) at 12-weeks follow-up. Marginal cost-effectiveness for each condition was calculated. Other similar randomised trials were also meta-analysed. RESULTS: The screening rate for two letters was 8.5% (OR=1.61, 95% CI 1.08-2.40) and 7.8% (OR=1.46, 95% CI 0.97-2.18) for one letter plus a telephone call, compared with 5.5% for standard practice (one letter) (OR=1.00). The response rate in the one letter plus a phone call group was 13.3% (OR=2.65, 95% CI 1.76-4.00) for women where a phone number was located. CONCLUSION: Initial screening rates after a 12-week follow-up were significantly higher in the women receiving a second invitation letter, compared with standard practice (one letter). Marginal cost-effectiveness favoured the two-letter approach. IMPLICATIONS: A follow-up invitation letter is more cost-effective than one invitation letter plus a follow-up telephone call in the BSNSW program. However, an invitation letter plus follow-up phone call is more cost-effective in recruiting women to BSNSW only if a phone number is located.     

Participation in breast cancer screening: randomised controlled trials of doctors' letters and of telephone reminders

Abstract: The study used a randomised controlled trial to find out whether supporting letters from general practitioners accompanying the invitations from a screening centre affected participation in a population-based breast cancer screening program for women aged 50 to 64. A further randomised controlled trial compared the effect of postal reminders with telephone reminders for women who did not respond to an initial invitation to participate in the program. There were 482 women in the first trial and 641 in the second. Excluding women who were ineligible or could not be contacted, participation in screening was 71 per cent in the group which received letters from their general practitioners compared with 62 per cent in the group which did not receive letters (P = 0.059). In the group that received letters, 56 per cent were screened without a reminder compared with 43 per cent of the group that did not receive letters (P = 0.01). Fewer women who received letters from their general practitioners declined the invitation to be screened (P = 0.048). In the second trial, there was no difference in participation between the group receiving telephone reminders and the group receiving postal reminders. As in breast cancer screening programs in other countries, general practitioner endorsement of invitations increased participation in breast cancer screening. Postal reminders were as effective as telephone reminders in encouraging women who did not respond to an initial invitation to participate in screening.     

Using the Internet to conduct surveys of health professionals: a valid alternative?

OBJECTIVE: The purpose of this study was to examine whether Internet-based surveys of health professionals can provide a valid alternative to traditional survey methods. METHODS: (i) Systematic review of published Internet-based surveys of health professionals focusing on criteria of external validity, specifically sample representativeness and response bias. (ii) Internet-based survey of GPs, exploring attitudes about using an Internet-based decision support system for the management of familial cancer. RESULTS: The systematic review identified 17 Internet-based surveys of health professionals. Whilst most studies sampled from professional e-directories, some studies drew on unknown denominator populations by placing survey questionnaires on open web sites or electronic discussion groups. Twelve studies reported response rates, which ranged from nine to 94%. Sending follow-up reminders resulted in a substantial increase in response rates. In our own survey of GPs, a total of 268 GPs participated (adjusted response rate = 52.4%) after five e-mail reminders. A further 72 GPs responded to a brief telephone survey of non-respondents. Respondents to the Internet survey were more likely to be male and had significantly greater intentions to use Internet-based decision support than non-respondents. CONCLUSIONS: Internet-based surveys provide an attractive alternative to postal and telephone surveys of health professionals, but they raise important technical and methodological issues which should be carefully considered before widespread implementation. The major obstacle is external validity, and specifically how to obtain a representative sample and adequate response rate. Controlled access to a national list of NHSnet e-mail addresses of health professionals could provide a solution.     

A comparison of general practitioner response rates to electronic and postal surveys in the setting of the National STI Prevention Program

Objective: To compare the response rates achieved for an online survey with a postal survey of general practitioners (GPs) as a method to evaluate the National STI Prevention Program. Methods: All GPs in Australia were asked to complete an online survey. A further sample of 509 GPs were asked to complete a postal survey. Response rates to both recruitment methods were compared. The demographic characteristics of responders were compared to the entire GP population of Australia. Results: Twenty GPs completed the online survey (response rate <0.1%). Sixty‐three GPs completed the postal survey (response rate 12.4%). The demographic characteristics of those responding to the postal survey showed no statistically significant difference compared to the general GP population. Conclusion: Our postal survey had a higher response rate than the online survey. Our response to the postal survey was lower than other similar studies and is likely to be due to a lack of incentives and follow‐up. Even with the low response rate it appears that postal surveys can provide a good representation of the overall population. Implications: Despite growing use of online surveys, postal surveys should still be the method of choice whenever possible. Postal surveys should include incentives and further follow‐up of the initial recruitment should be conducted.     

Women in an infertility survey responded more by mail but preferred a choice: randomized controlled trial

ObjectivesTo evaluate three modes of questionnaire completion (online, mail, and telephone) and a choice group for a questionnaire survey with a sensitive topic.Study Design and SettingA randomized trial of alternative completion methods (online, paper [mail], telephonic interview, and a choice of the three modes) for a survey about fertility problems embedded within a population-based cross-sectional survey of reproduction among women living in England and Wales.ResultsFrom an initial cohort of 21,036 women who were sent a screening questionnaire, 4,559 responded. A total of 699 women reported fertility problems, of whom 593 (85%) agreed to participate. A total of 521 (75%) of the 699 women then completed the survey. Compared with the “mail” arm, those allocated to telephonic interview were less likely to agree to participate initially (odds ratio [OR], 0.41; 95% confidence interval [CI]: 0.22, 0.74) but were ultimately more likely to complete the questionnaire (OR, 2.20; 95% CI: 1.01, 4.80). Overall, those allocated to the choice arm were most likely to go through to completion (80% vs. 77% mail, 72% telephone, and 68% online groups). In the choice arm, women showed a clear preference for mail (59% vs. 37% choosing online and 3% telephonic response).ConclusionsOnline surveys are a viable alternative to mailed questionnaires but were not as popular as mail in this study population. Response can possibly be increased by offering women a choice of response modes.     

Electronic mail was not better than postal mail for surveying residents and faculty

OBJECTIVE: To compare response rate, time to response, and data quality of electronic and postal surveys in the setting of postgraduate medical education. STUDY DESIGN AND SETTING: A randomized controlled trial in a university-based internal medicine residency program. We randomized 119 residents and 83 faculty to an electronic versus a postal survey with up to two reminders and measured response rate, time to response, and data quality. RESULTS: For residents, the e-survey resulted in a lower response rate than the postal survey (63.3% versus 79.7%; difference -16.3%, 95% confidence interval (95% CI) -32.3% to -0.4%%; P=.049), but a shorter mean response time, by 3.8 days (95% CI 0.2-7.4; P=.042). For faculty, the e-survey did not result in a significantly lower response rate than the postal survey (85.4% vs. 81.0%; difference 4.4%, 95% CI -11.7 to 20.5%; P=.591), but resulted in a shorter average response time, by 8.4 days (95% CI 4.4 to 12.4; P < 0.001). There were no differences in the quality of data or responses to the survey between the two methods. CONCLUSION: E-surveys were not superior to postal surveys in terms of response rate, but resulted in shorter time to response and equivalent data quality.     

Telephone reminders are a cost effective way to improve responses in postal health surveys

STUDY OBJECTIVE: To assess the effectiveness of a telephone reminder in increasing responses to postal surveys and to calculate the differential costs per completed questionnaire. DESIGN: Randomised controlled trial. SETTING: Australian university and rehabilitation medicine practice. PARTICIPANTS: The trial was conducted in 1999 among the 143 non-respondents to a questionnaire about work related neck and upper body disorders. The questionnaire was sent to two Australian female samples: 200 office workers (Sample A) and 92 former rehabilitation medicine patients (Sample B). A reminder letter, another copy of the questionnaire and a final letter were sent at two week intervals. Half of the non-respondents within each sample were randomly selected to receive a telephone reminder just after the second mailout of the questionnaire. All direct costs were calculated. MAIN RESULTS: Responses were significantly higher among those who received the telephone reminder intervention (relative risk 2.54, 95% confidence intervals 1.43 to 4.52). Analysed by intention to phone, 47% of non-respondents in Sample A and 38% in Sample B returned a complete questionnaire after the intervention, compared with 21% and 10%, respectively, in the control groups. For the 112 women (combined samples) who returned completed questionnaires before randomisation, the average cost per respondent was AUD14. There was a higher total cost for the intervention groups (AUD851 versus AUD386 for controls), but the significantly higher number of additional completed responses (31 versus 12) resulted in a 15% lower marginal cost per completed questionnaire in those groups. CONCLUSION: Telephone reminders are cost effective in improving responses to postal surveys.     

The cost-effectiveness of cash versus lottery incentives for a web-based, stated-preference community survey

We present the results of a randomized experiment to test the effectiveness and cost-effectiveness of response incentives for a stated-preference survey of a general community population. The survey was administered using a mixed-mode approach, in which community members were invited to participate using a traditional mailed letter using contact information for a representative sample of the community; but individuals completed the survey via the web, which exploited the advantages of electronic capture. Individuals were randomized to four incentive groups: (a) no incentive, (b) prepaid cash incentive ($2), (c) a low lottery (10 prizes of $25) and (d) a high lottery (2 prizes of $250). Letters of invitation were mailed to 3,000 individuals. In total, 405 individuals (14.4%) contacted the website and 277 (9.8%) provided complete responses. The prepaid cash incentive generated the highest contact and response rates (23.3 and 17.3%, respectively), and no incentive generated the lowest (9.1 and 5.7%, respectively). The high lottery, however, was the most cost-effective incentive for obtaining completed surveys: compared with no incentive, the incremental cost-effectiveness ratio (ICER) per completed survey for high lottery was $13.89; for prepaid cash, the ICER was $18.29. This finding suggests that the preferred response incentive for community-based, stated-preference surveys is a lottery with a small number of large prizes.     

Adolescent preventive health visits: a comparison of two invitation protocols

BACKGROUND: Adolescent health care in family practice at times creates conflicting responsibilities for parents and their teenagers. In the context of a new adolescent preventive health program in a family practice setting, we compared attendance rates using two invitation protocols, the protocols differing in their emphasis on adolescent autonomy vs parental responsibility. METHODS: One hundred six teenagers in the seventh and tenth grades were invited for preventive health visits with the family nurse and physician using two protocols. Protocol 1 involved obtaining parental consent before approaching the adolescent. With protocol 2, an invitation letter and parental consent form were mailed to the teenager, while a letter of explanation was sent concurrently to the parents. In each case, the letter of invitation was followed up by a telephone call for those who did not respond. The spontaneous response rate (a positive response after receiving the letter), agreement to attend rate (a positive response after receiving the letter or being telephoned), and the attendance rate were determined according to grade, sex, and protocol. RESULTS: The spontaneous response rate was 21%, the agreement to attend rate was 75%, and the attendance rate was 44%. Attendance rates were higher for the girls compared with the boys (54% vs 35%, P = .08) and for the seventh graders compared with the tenth graders (53% vs 31%, P = .03). The spontaneous response rate was lower among the tenth graders using protocol 2 (8% vs 37.5% with protocol 1, P = .04), while the agreement to attend rate and attendance rate did not differ for the two protocols. CONCLUSIONS: Nearly one half of this population of adolescents attended preventive health visits at the family nurse's and physician's initiative. A follow-up telephone call after the initial written invitation resulted in increased participation, while approaching the teenager or parent initially did not make a difference in attendance. This pilot study shows the potential for initiating an adolescent health program in the family practice setting.     

Not another questionnaire! Maximizing the response rate, predicting non-response and assessing non-response bias in postal questionnaire studies of GPs

BACKGROUND: Non-response is an important potential source of bias in survey research. With evidence of falling response rates from GPs, it is of increasing importance when undertaking postal questionnaire surveys of GPs to seek to maximize response rates and evaluate the potential for non-response bias. OBJECTIVES: Our aim was to investigate the effectiveness of follow-up procedures when undertaking a postal questionnaire study of GPs, the use of publicly available data in assessing non-response bias and the development of regression models predicting responder behaviour. METHOD: A postal questionnaire study was carried out of a random sample of 600 GPs in Wales concerning their training and knowledge in palliative care. RESULTS: A cumulative response rate graph permitted optimal timing of follow-up mailings: a final response rate of 67.6% was achieved. Differences were found between responders and non-responders on several parameters and between sample and population on some parameters: some of these may bias the sample data. Logistic regression analysis indicated medical school of qualification and current membership of the Royal College of General Practitioners to be the only significant predictors of responders. Late responders were significantly more likely to have been qualified for longer. CONCLUSIONS: This study has several implications for future postal questionnaire studies of GPs. The optimal timing of reminders may be judged from plotting the cumulative response rate: it is worth sending at least three reminders. There are few parameters that significantly predict GPs who are unlikely to respond; more of these may be included in the sample, or they may be targeted for special attention. Publicly available data may be used readily in the analysis of non-response bias and generalizability.     

Increasing response rates of gynaecologists to a survey: a randomised trial of telephone prompts

Abstract: This study used a randomised controlled trial to evaluate the effectiveness of telephone prompts to increase the response rate of gynaecologists to a survey. A nine-page questionnaire about management of women with abnormal Pap smears was mailed to Fellows of the Royal Australian College of Obstetricians and Gynaecologists. A random sample was contacted in advance by telephone, and nonrespondents received a telephone reminder 14 days after the initial mail-out. The control group was mailed questionnaires and a reminder letter conventionally. On Day 25, nonresponders in both groups were mailed a second questionnaire. Of the intervention group, 84 per cent responded to the survey compared to 69 per cent of the control group (P = 0.005). A telephone call in anticipation of a mailed questionnaire is highly recommended.     

A very low response rate in an on-line survey of medical practitioners

OBJECTIVE: To report on the response rate achieved in a survey of medical practitioners and discuss the reasons for it. METHOD: An on-line (internet-based) survey of all 609 registered pharmacotherapy prescribers in Victoria and Queensland; invitations to participate were sent by mail in late April 2007, and one reminder letter in late May 2007. RESULTS: Six hundred and nine invitation letters were mailed, nine were returned to sender, and 52 questionnaires completed, making the overall response rate 52/600 = 8.7%. The response rate in Queensland was 13.2% (16/121), and in Victoria 7.5% (36/479). CONCLUSIONS: Despite utilising sound techniques, our response rate was much lower than those achieved in recent Australian paper-based surveys of medical practitioners. It is possible that the issue being addressed (injecting-related injuries and diseases) was not of high priority for many invitees, leading to reduced response. IMPLICATIONS: On-line surveys are not yet an effective method of collecting data from Australian medical practitioners; researchers should continue to use paper questionnaires for maximum response.     

Format and readability of an enhanced invitation letter did not affect participation rates in a cancer registry-based study: a randomized controlled trial

ObjectivesTo assess the effectiveness of an “enhanced” invitation letter in increasing participation in an Australian cancer registry-based study and assess the representativeness of the study sample.Study Design and SettingEight hundred hematological cancer survivors, diagnosed within the last 3 years and aged 18–80 years at recruitment, were selected from one Australian state-based cancer registry. Half were randomly allocated to receive the standard invitation letter (control group). The remaining half received a modified invitation letter, incorporating content and design characteristics recommended to improve written communication (intervention group).ResultsOf the 732 eligible survivors, 268 (37%) returned a completed survey. There was no difference in participation between the intervention (n = 131, 36%) and control groups (n = 137, 38%; P = 0.53). Participants were representative of the population for characteristics assessed, except for age group at diagnosis. Survivors 50 years or older at diagnosis had higher odds of returning a completed survey, 50–59 (odds ratio [OR]: 2.53; 95% confidence interval [CI]: 1.47, 4.35), 60–69 (OR: 2.69; 95% CI: 1.58–4.58), and 70–80 (OR: 1.90; 95% CI: 1.07–3.35), than survivors aged 15–39 years at diagnosis.ConclusionAn enhanced invitation letter was not effective in increasing participation of hematological cancer survivors in an Australian cancer registry study. The study sample was moderately representative on variables assessed, with age group at diagnosis the only variable associated with participation. Research should evaluate strategies to increase participation in registry studies and focus on tailoring techniques to patient's age.     

Ethnicity, but not cancer family history, is related to response to a population-based mailed questionnaire

PURPOSE: To determine if family history and ethnic background are factors affecting response to a mailed cancer family history questionnaire from the Ontario Familial Breast Cancer Registry. METHODS: Individuals diagnosed with primary invasive breast carcinomas (probands) were mailed a family history questionnaire, the first contact in a multi-stage process. This questionnaire obtained cancer family history and ethnicity data. After one month, a follow up telephone call was made to those who did not return this questionnaire and attempts were made to ask similar questions by telephone interview. Characteristics of those responding to the mailed questionnaire were compared to those who responded to the telephone interview only. RESULTS: 339 probands were included in this study: 242 returned a mailed version of the questionnaire; 57 completed the questionnaire over the phone. Cancer family history/genetic risk criteria was not significantly related to type of response. Probands identifying themselves as visible minorities were significantly less likely to respond to the mailed questionnaire than the telephone interview (11.6% vs. 22.8%, P=0.03). CONCLUSIONS: Having a family history of cancer did not appear to influence response to a mailed questionnaire, but those reporting an ethnic/racial background other than White were more likely to respond to a telephone interview.     

Nonresponse bias and cost-effectiveness in a Norwegian survey of family physicians

Low response rates are a common problem in surveys of family physicians leading to uncertainty about the validity of results. In this study, the authors examined the association between multiple reminders and nonresponse bias, survey estimates and costs in a survey of family physicians in Norway (N = 3,463). After three postal reminders and one telephone follow-up, the response rate was 65.9%. They analyzed differences in nine demographic and practice variables between respondents and nonrespondents, the effect of nonresponse bias on survey estimates, and the cost-effectiveness of each reminder. Statistically significant differences between respondents and nonrespondents were found for six variables. However, demographic and practice variables had little association with the main outcome variables, and the overall survey estimates changed little with additional reminders. In addition, the cost-effectiveness of the final reminders was poor.     

Comparability of skin screening histories obtained by telephone interviews and mailed questionnaires: a randomized crossover study

The comparability of information collected through telephone interviews and information collected through mailed questionnaires has not been well studied. As part of the first phase of a randomized controlled trial of population screening for melanoma in Queensland, Australia, the authors compared histories of skin examination reported in telephone interviews and self-administered mailed questionnaires. A total of 1,270 subjects each completed a telephone interview and a mailed questionnaire 1 month apart in 1999; 564 subjects received the interview first, and 706 received the mailed questionnaire first. Agreement between the two methods was 91.2% and 88.6% for whole-body skin examination by a physician in the last 12 months and the last 3 years, respectively, and 81.9% for whole-body skin self-examination in the last 12 months. Agreement was lower for "any" skin self-examination. Agreement between the two methods was similar regardless of whether the interview or the questionnaire was administered first. Missing data were less frequent for interviews (0.5%) than for mailed questionnaires (3.8%). Costs were estimated at A$9.55 (US$6.21) per completed interview and A$3.01 (US$1.96) per questionnaire. The similarity of results obtained using telephone interviews and mailed questionnaires, coupled with the substantially higher cost of telephone interviews, suggests that self-administered mailed questionnaires are an appropriate method of assessing this health behavior.     

Prepayment was superior to postpayment cash incentives in a randomized postal survey among physicians

OBJECTIVE: Improving response rates, particularly among physicians, is important to minimize nonresponder bias and increase the effective sample size in epidemiologic research. We conducted a randomized trial to examine the impact of prepayment vs. postpayment incentives on response rates. STUDY DESIGN AND SETTING: Self-completion postal questionnaires were mailed to 949 physicians who were respondents to an earlier survey and representative of the general physician population in Hong Kong. These physicians were randomly allocated to receive a HK dollar 20 cash prepayment incentive that accompanied the survey (n=474) or a postpayment reward of the same amount on receipt of the completed questionnaire (n=475). RESULTS: The final prepayment response rate was 82.9%, compared with 72.5% in the postpayment arm (P < .001). Of the eight alternative incentive and follow-up strategies evaluated, three lie on the efficiency frontier (i.e., not dominated), including postpayment with three mailings at HK dollar 42.7, prepayment with three mailings at HK dollar 66.5 and prepayment with three mailings and telephone follow-up at HK dollar 112.1 per responder recruited (US dollar 1=HK dollar 7.8). CONCLUSION: The findings demonstrate that prepayment cash incentives are superior to postpayment of the equivalent amount in improving response rates among a representative sample of Hong Kong physicians. Further research should concentrate on confirming the generalizability of these findings in other health care occupation groups and settings.     

Contribution of follow-up of nonresponders to prevalence and risk estimates: a Norwegian respiratory health survey

Achieving a high response rate is often expensive and time consuming. Does an extensive survey effort change prevalence estimates and exposure-disease relations? In 1998-1999, the authors conducted a population-based respiratory health survey in two Norwegian counties (Oslo, n = 20,000; Hordaland, n = 5,000) of a random sample of the adult population aged 15-70 years. A postal questionnaire was mailed, with as many as two reminder letters. A 25% random sample of postal nonresponders was contacted by ordinary or cellular telephone. Cumulative response rates after the first mailing, first reminder, second reminder, and telephone follow-up were 42.7%, 60.7%, 68.3%, and 79.9%, respectively. Compared with initial postal responders, responders at later stages were younger, and more were male. Late responders had a lower educational level, were more often smokers, and were occupationally exposed to dusts or gases more frequently. After the authors adjusted for these factors, late responders were found to have less asthma, attacks of breathlessness, and hay fever. Hardly any changes in prevalences and odds ratios were noted when initial responders were compared with all responders. The additional contribution of sending reminder letters and conducting a telephone follow-up to prevalence estimates and the exposure-disease relation was small. A single letter would have produced nearly the same prevalence estimates and risk factor associations.