A randomised controlled trial to determine the effect on response of including a lottery incentive in health surveys [ISRCTN32203485]

Background

Postal questionnaires are an economical and simple method of data collection for research purposes but are subject to non-response bias. Several studies have explored the effect of monetary and non-monetary incentives on response. Recent meta-analyses conclude that financial incentives are an effective way of increasing response rates. However, large surveys rarely have the resources to reward individual participants. Three previous papers report on the effectiveness of lottery incentives with contradictory results. This study aimed to determine the effect of including a lottery-style incentive on response rates to a postal health survey.

Methods

Randomised controlled trial. Setting: North and West Birmingham. 8,645 patients aged 18 or over randomly selected from registers of eight general practices (family physician practices). Intervention: Inclusion of a flyer and letter with a health questionnaire informing patients that returned questionnaires would be entered into a lottery-style draw for £100 of gift vouchers. Control: Health questionnaire accompanied only by standard letter of explanation. Main outcome measures: Response rate and completion rate to questionnaire.

Results

5,209 individuals responded with identical rates in both groups (62.1%). Practice, patient age, sex and Townsend score (a postcode based deprivation measure) were identified as predictive of response, with higher response related to older age, being female and living in an area with a lower Townsend score (less deprived).

Conclusion

This RCT, using a large community based sample, found that the offer of entry into a lottery style draw for £100 of High Street vouchers has no effect on response rates to a postal health questionnaire.     

Efficacy of GP referral of insufficiently active patients for expert physical activity counseling: protocol for a pragmatic randomized trial (The NewCOACH trial)

Background

The Scottish Naloxone Programme aims to reduce Scotland’s high number of drug-related deaths (DRDs) caused by opiate overdose. It is currently implemented through specialist drug services but General Practitioners (GPs) are likely to have contact with drug using patients and their families and are therefore in an ideal position to direct them to naloxone schemes, or provide it themselves. This research gathered baseline data on GP’s knowledge of and willingness to be involved in DRD prevention, including naloxone administration, prior to the implementation of primary care based delivery.

Methods

Mixed methods were used comprising a quantitative, postal survey and qualitative telephone interviews. A questionnaire was sent to 500 GPs across Scotland. An initial mailing was followed by a reminder. A shortened questionnaire containing seven key questions was posted as a final reminder. Telephone interviews were conducted with 17 GPs covering a range of demographic characteristics and drug user experience.

Results

A response rate of 55% (240/439) was achieved. There was some awareness of the naloxone programme but little involvement (3.3%), 9% currently provided routine overdose prevention, there was little involvement in displaying overdose prevention information (<20%). Knowledge of DRD risk was mixed. There was tentative willingness to be involved in naloxone prescribing with half of respondents willing to provide this to drug users or friends/family. However half were uncertain GP based naloxone provision was essential to reduce DRDs. Factors enabling naloxone distribution were: evidence of effectiveness, appropriate training, and adding to the local formulary. Interviewees had limited awareness of what naloxone distribution in primary care may involve and considered naloxone supply as a specialist service rather than a core GP role. Wider attitudinal barriers to involvement with this group were expressed.

Conclusions

There was poor awareness of the Scottish National Naloxone Programme in participants. Results indicated GPs did not currently feel sufficiently skilled or knowledgeable to be involved in naloxone provision. Appropriate training was identified as a key requirement.     

Collection of vaccination data in the German National Cohort

Background

Data about the vaccination status of participants are required in epidemiological cohort studies whenever infection or immunity is considered as potential exposure or outcome. Within Pretest 2 of the German National Cohort (GNC) we therefore investigated the acceptance and feasibility of extracting vaccination status from vaccination certificates provided by the participants of the study.

Methods

This study was conducted in three study centers (Bremen, n?=?73; Hamburg, n?=?200; Hannover, n?=?193). In order to test if an additional reminder would prevent participants from forgetting their vaccination certificates at home persons willing to participate in Pretest 2 were randomly assigned to one of three invitation groups (IG). About one third of the participants received either no further reminder (IG1), a reminder card together with the appointment letter (IG2) or a separate reminder card 4 days before the appointment (IG3). At the study center, vaccination data were scanned or copied and entered into a database using a unique identification number. Participants were also asked to fill in a short questionnaire to assess the completeness of the provided vaccination data. Additionally, in one of the three participating study centers, general practitioners (GP) were asked to provide vaccination data from their records following respective participants’ consent. Finally, we compared the influenza data from the vaccination certificates with the influenza data obtained from participants in Pretest 2 by use of a self-administered questionnaire (ID-Screen).

Results

Due to different starting dates of the study the intended reminder procedure was implemented only in Hamburg and Hannover. In Hamburg, significantly more vaccination certificates were submitted by the group which received the reminder card separately 4 days before the examination (IG3) compared to IG1 and IG2 (p?=?0.04). In Hannover, in contrast, most vaccination certificates were brought by those who received the reminder card together with the appointment letter. Overall, the use of a reminder card had a positive but not significant effect as 89?% (185/209) of participants who received the reminder card submitted vaccination data versus 81?% (84/104) of participants who did not receive any reminder card (p?=?0.06). Of all Pretest 2 participants in Hannover, 62?% (120/193) gave written consent for data collection by the GPs. In total, 114 practices were contacted of which 49 (43?%) sent vaccination data. All in all, 360 vaccination certificates with 5065 documented vaccinations were entered into a database, of which 4830 (95?%) were valid for analysis covering a period from 1946 to 2012. The comparison of influenza vaccination data from vaccination certificates to the remembered data from a self-completed questionnaire showed an agreement of data in 46?% (84/184) of cases (Kappa?=?0.48). Influenza vaccinations were underreported in 4?% (7/170) of self-completed questionnaires.

Conclusion

The reliable documentation of vaccinations within the context of the GNC proved to be feasible and thus recommendable at a large scale within the GNC as participants showed high willingness and compliance in providing available vaccination documents. An additional validation by means of documents provided by physicians seems to be possible for more than a quarter of participants. In order to maximize the likelyhood of participants’ of bringing their vaccination certificates it would be sufficient to send a reminder card together with the appointment letter.     

Effect of prize draw incentive on the response rate to a postal survey of obstetricians and gynaecologists: A randomised controlled trial. [ISRCTN32823119]

Background

Response rates to postal questionnaires are falling and this threatens the external validity of survey findings. We wanted to establish whether the incentive of being entered into a prize draw to win a personal digital assistant (PDA) would increase the response rate for a national survey of consultant obstetricians and gynaecologists.

Methods

A randomised controlled trial was conducted. This involved sending a postal questionnaire to all Consultant Obstetricians and Gynaecologists in the United Kingdom. Recipients were randomised to receiving a questionnaire offering a prize draw incentive (on response) or no such incentive.

Results

The response rate for recipients offered the prize incentive was 64% (461/716) and 62% (429/694) in the no incentive group (relative rate of response 1.04, 95% CI 0.96 - 1.13)

Conclusion

The offer of a prize draw incentive to win a PDA did not significantly increase response rates to a national questionnaire survey of consultant obstetricians and gynaecologists.     

Telephone reminders are a cost effective way to improve responses in postal health surveys

STUDY OBJECTIVE: To assess the effectiveness of a telephone reminder in increasing responses to postal surveys and to calculate the differential costs per completed questionnaire. DESIGN: Randomised controlled trial. SETTING: Australian university and rehabilitation medicine practice. PARTICIPANTS: The trial was conducted in 1999 among the 143 non-respondents to a questionnaire about work related neck and upper body disorders. The questionnaire was sent to two Australian female samples: 200 office workers (Sample A) and 92 former rehabilitation medicine patients (Sample B). A reminder letter, another copy of the questionnaire and a final letter were sent at two week intervals. Half of the non-respondents within each sample were randomly selected to receive a telephone reminder just after the second mailout of the questionnaire. All direct costs were calculated. MAIN RESULTS: Responses were significantly higher among those who received the telephone reminder intervention (relative risk 2.54, 95% confidence intervals 1.43 to 4.52). Analysed by intention to phone, 47% of non-respondents in Sample A and 38% in Sample B returned a complete questionnaire after the intervention, compared with 21% and 10%, respectively, in the control groups. For the 112 women (combined samples) who returned completed questionnaires before randomisation, the average cost per respondent was AUD14. There was a higher total cost for the intervention groups (AUD851 versus AUD386 for controls), but the significantly higher number of additional completed responses (31 versus 12) resulted in a 15% lower marginal cost per completed questionnaire in those groups. CONCLUSION: Telephone reminders are cost effective in improving responses to postal surveys.     

International Questionnaire Postal Response Rate: An experiment comparing no return postage to provision of International Postage Vouchers - "Coupon-Réponse International"

Background

Previous studies of methods to increase response rates on international postal surveys did not include providing return postage. We provided International Postage Vouchers - "Coupon-Réponse International" to cover this. The objective of this study was to see if these International Postage Vouchers had an effect on response rates.

Methods

Between-groups, randomized, after-only

Results

There was no difference in response rates between the group that received International Postage Vouchers and the group that did not. (p = 0.23)

Conclusion

International Postage Vouchers - "Coupon-Réponse International" have no effect on response rates for international postal surveys.     

Interviewer BMI effects on under- and over-reporting of restrained eating: evidence from a national Dutch face-to-face survey and a postal follow-up

Objectives

To determine the effect of interviewer BMI on self-reported restrained eating in a face-to-face survey and to examine under- and over-reporting using the face-to face study and a postal follow-up.

Methods

A sample of 1,212 Dutch adults was assigned to 98 interviewers with different BMI who administered an eating questionnaire. To further evaluate misreporting a mail follow-up was conducted among 504 participants. Data were analyzed using two-level hierarchical models.

Results

Interviewer BMI had a positive effect on restrained eating. Normal weight and pre-obese interviewers obtained valid responses, underweight interviewers stimulated under-reporting whereas obese interviewers triggered over-reporting.

Conclusion

In face-to-face interviews self-reported dietary restraint is distorted by interviewer BMI. This result has implications for public health surveys, the more so given the expanding obesity epidemic.     

Using different approaches to conducting postal questionnaires affected response rates and cost-efficiency

Objective

To compare three different approaches for consent in postal questionnaire in terms of response rate, time consumption, and cost-efficiency, and to collect a demographic questionnaire for dropout analyses.

Study Design and Setting

Population survey in Sweden. Mothers and fathers (n = 600) of three hundred 3-year olds were divided into three groups. One group was asked to Actively Agree to participate in a cover letter and send consent back to receive the main questionnaire. The second group received the cover letter, the consent, and the main questionnaire in the initial mailings, Direct Delivery. The third group received the cover letter and consent form in which they were asked to Actively Decline participation within 7 days if they did not want to participate. Otherwise, they were sent the main questionnaire. All parents were asked to fill in a demographic questionnaire regardless of whether they wanted to complete the main questionnaire.

Results

The highest response rate was in the Actively Decline mode. The cost-efficiency for this approach was 1.52 compared with Direct Delivery and 1.29 compared with Actively Agree.

Conclusion

Researchers can improve the response rate, time consumption, and cost-efficiency and obtain a demographic questionnaire for dropout analysis by using the Actively Decline approach for postal questionnaires.     

Management of children's urinary tract infections in Dutch family practice: a cohort study

Background

Although cardiovascular prediction rules are recommended by guidelines to evaluate global cardiovascular risk for primary prevention, they are rarely used in primary care. Little is known about barriers for application. The objective of this study was to evaluate barriers impeding the application of cardiovascular prediction rules in primary prevention.

Methods

We performed a postal survey among general physicians in two Swiss Cantons by a purpose designed questionnaire.

Results

356 of 772 dispatched questionnaires were returned (response rate 49.3%). About three quarters (74%) of general physicians rarely or never use cardiovascular prediction rules. Most often stated barriers to apply prediction rules among rarely- or never-users are doubts concerning over-simplification of risk assessment using these instruments (58%) and potential risk of (medical) over-treatment (54%). 57% report that the numerical information resulting from prediction rules is often not helpful for decision-making in practice.

Conclusion

If regular application of cardiovascular prediction rules in primary care is in demand additional interventions are needed to increase acceptance of these tools for patient management among general physicians.     

A low response rate does not necessarily indicate non-response bias in gastroenterology survey research: a population-based study

Aim

To estimate the potential for response bias in standard mailed questionnaires used in surveys of GI symptoms in a community.

Subjects and methods

Validated self-report tools have been developed to measure functional gastrointestinal (GI) disorders but response rates in community surveys have been rapidly declining in many parts of the world. Whether a lower community response rate introduces significant response bias in GI survey research is unknown. A questionnaire was mailed to a total of 5,069 randomly selected subjects. The overall response rate was 52 %. A random sample of 723 of these subjects (428 responders and 295 non-responders, stratified by age and gender) was selected for medical record abstraction (including both inpatient and outpatient history).

Results

The odds for response increased in those with a higher body mass index (odds ratio (OR):1.02 [95 % CI: 1.01, 1.03]), more health care seeking behavior for non-GI problems (OR: 1.97 [95 % CI: 1.43, 2.72]), and for those who had responded to a previous survey (OR: 4.84 [95 % CI: 2.84, 8.26]). Responder status was not significantly associated with any GI symptoms or a diagnosis of GI or non-GI disease (with two exceptions, diverticulosis and skin disease).

Conclusions

Despite a response rate of only 52 %, the results of a community-based GI survey do not appear to be impacted by non-response bias in a major way. A low survey response rate does not necessarily indicate non-response bias.     

Screening of the pelvic organ prolapse without a physical examination; (a community based study)

Background

The attendance rate in Estonian cervical cancer screening programme is too low therefore the programme is hardly effective. A cross-sectional population based survey was performed to identify awareness of cervical cancer risk factors, reasons why women do not want to participate in cervical screening programme and wishes for better organisation of the programme.

Method

An anonymous questionnaire with a covering letter and a prepaid envelope was sent together with the screening invitation to 2942 randomly selected women. Results are based on the analysis of 1054 (36%) returned questionnaires.

Results

Main reasons for non-participation in the national screening programme were a recent visit to a gynaecologist (42.3%), fear to give a Pap-smear (14.3%), long appointment queues (12.9%) and unsuitable reception hours (11.8%). Fear to give a Pap-smear was higher among women aged 30 and 35 than 50 and 55 (RR 1.46; 95% CI: 0.82-2.59) and women with one or no deliveries (RR 1.56, 95% CI: 0.94-2.58). In general, awareness of cervical cancer risk factors is poor and it does not depend on socio-demographic factors. Awareness of screening was higher among Estonians than Russians (RR 1.64, 95% CI: 1.46-1.86). Most women prefer to receive information about screening from personally mailed invitation letters (74.8%).

Conclusions

Women need more information about cervical cancer risk factors and the screening programme. They prefer personally addressed information sharing. Minority groups should be addressed in their own language. A better collaboration with service providers and discouraging smears outside the programme are also required.     

Identification of adults with symptoms suggestive of obstructive airways disease: Validation of a postal respiratory questionnaire

Background

Two simples scoring systems for a self-completed postal respiratory questionnaire were developed to identify adults who may have obstructive airways disease. The objective of this study was to validate these scoring systems.

Method

A two-stage design was used. All adults in two practice populations were sent the questionnaire and a stratified random sample of respondents was selected to undergo full clinical evaluation. Three respiratory physicians reviewed the results of each evaluation. A majority decision was reached as to whether the subject merited a trial of obstructive airways disease medication. This clinical decision was compared with two scoring systems based on the questionnaire in order to determine their positive predictive value, sensitivity and specificity.

Results

The PPV (positive predictive value) of the first scoring system was 75.1% (95% CI 68.6–82.3), whilst that of the second system was 82.3% (95% CI 75.9–89.2). The more stringent second system had the greater specificity, 97.1% (95% CI 96.0–98.2) versus 95.3% (95% CI 94.0–96.7), but poorer sensitivity 46.9% (95% CI 33.0–66.8) versus 50.3% (95% CI 35.3–71.6).

Conclusion

This scoring system based on the number of symptoms/risk factors reported via a postal questionnaire could be used to identify adults who would benefit from a trial of treatment for obstructive airways disease.

     

Violence towards health care workers in a Public Health Care Facility in Italy: a repeated cross-sectional study

Background

Violence at work is one of the major concerns in health care activities. The aim of this study was to identify the prevalence of physical and non-physical violence in a general health care facility in Italy and to assess the relationship between violence and psychosocial factors, thereby providing a basis for appropriate intervention.

Methods

All health care workers from a public health care facility were invited to complete a questionnaire containing questions on workplace violence. Three questionnaire-based cross-sectional surveys were conducted. The response rate was 75 % in 2005, 71 % in 2007, and 94 % in 2009. The 2009 questionnaire contained the VIF (Violent Incident Form) for reporting violent incidents, the DCS (demand/control/support) model for job strain, the Colquitt 20 item questionnaire for perceived organizational justice, and the GHQ-12 General Health Questionnaire for the assessment of mental health.

Results

One out of ten workers reported physical assault, and one out of three exposure to non-physical violence in the workplace in the previous year. Nurses and physicians were the most exposed occupational categories, whereas the psychiatric and emergency departments were the services at greatest risk of violence. Workers exposed to non-physical violence were subject to high job strain, low support, low perceived organizational justice, and high psychological distress.

Conclusion

Our study shows that health care workers in an Italian local health care facility are exposed to violence. Workplace violence was associated with high demand and psychological disorders, while job control, social support and organizational justice were protective factors.     

Postal surveys of physicians gave superior response rates over telephone interviews in a randomized trial

BACKGROUND AND OBJECTIVES: To compare general practitioner (GP) response to a telephone interview with response to a postal survey with three reminders in a randomized controlled trial. METHODS: GPs were randomly assigned to either a telephone interview or a postal survey. GPs in the telephone group were mailed a letter of invitation and asked to undertake a telephone interview. GPs in the postal group were mailed a letter of invitation and questionnaire. Non-responders were sent up to three reminders, the final by registered post. Response rates were calculated for each group. RESULTS: 416 GPs were randomized to the telephone interview and 451 to the postal survey. Eighty-six in the telephone group and 30 in the postal were ineligible. One hundred thirty-four GPs completed the telephone interview with a response rate of 40.6% (95% confidence interval [CI]: 35.3%, 46.1%). Two hundred fifty-two GPs completed the postal survey with a response rate of 59.9% (95%CI: 55.0%, 64.6%). The difference in response was 19.3% (95%CI: 12.2%, 26.3%). CONCLUSIONS: These results show that postal surveys with three reminders can have superior response rates compared with a telephone interview.     

Factors Influencing Participation Rate in a Baseline Survey of a Genetic Cohort in Japan

Background

Although many studies have examined factors that influence the response to postal questionnaires, few have addressed baseline recruitment for cohort studies involving genetic analyses. The aim of this study was to describe the method used for a baseline survey, the Japan Multi-institutional Collaborative Cohort Study (J-MICC Study), in Saga Prefecture, and to examine the factors that might influence the recruitment of participants in such studies.

Methods

The Saga J-MICC Study is an ongoing population-based prospective cohort study of the genetic and environmental interactions associated with lifestyle-related disease. From 2005 through 2007, a total of 61 447 residents between the ages of 40 and 69 were invited by mail to participate in this study. The survey date and time were arranged by telephone.

Results

Among that population, 31 002 (50.5%) responded and 12 078 (19.7%) agreed to participate. A completed questionnaire and blood pressure and anthropometric data were collected from all participants; blood, DNA specimens, and accelerometer measures were obtained from the great majority of them. Female sex and older age were associated with a higher participation rate. In addition, the convenience of the survey location and the sending of a reminder significantly improved the participation rate (odds ratio, 1.3).

Conclusions

Our findings suggest that making the survey location as convenient as possible and sending a reminder can both substantially improve participation rate in population-based studies.Key words: population-based study, survey methodology, response rate, participation rate, reminder     

Impact of a postcard versus a questionnaire as a first reminder in a postal lifestyle survey.

STUDY OBJECTIVE--The study aimed to consider the impact of two different types of reminder on response rates and costs in a postal survey. DESIGN--The study was a cross sectional survey. A self-completion lifestyle questionnaire was used. Those who did not respond after the initial mailing were randomly allocated to receive either a postcard or questionnaire as a first reminder. All outstanding non-responders received a questionnaire as a second reminder. SUBJECTS--A representative sample of 698 adults aged 16-70 was used, drawn from a family health services authority register. MAIN RESULTS--Postcard reminders were as effective as questionnaire reminders in increasing response whether one or two reminders are sent. The costs per response were calculated. Two questionnaires as reminders were found to be 1.7 times more expensive than a postcard plus questionnaire. Including the initial mailing, the cost per response using all questionnaires was 1.3 times the cost when a postcard was used for the first reminder. CONCLUSIONS--To increase the response to a postal survey effectively and economically, two reminders should be sent--first a postcard and then a questionnaire.     

Maximising Responses to Discrete Choice Experiments

Objective

To identify any differences in response and completion rates across two versions of a questionnaire, in order to determine the trade-off between a potentially higher response rate (from a short questionnaire) and a greater level of information from each respondent (from a long questionnaire).

Methods

This was a randomised trial to determine whether response rates and/or results differ between questionnaires containing different numbers of choices: a short version capable of estimating main effects only and a longer version capable of estimating two-way interactions, provided certain assumptions hold. Best-worst scaling was the form of discrete choice experimentation used. Data were collected by post and analysed in terms of response rates, completion rates and differences in mean utilities.

Results

Fifty-three percent of individuals approached agreed to take part. From these, the response to the long questionnaire was 83.2% and the short questionnaire was 85.1% (difference 1.9%, 95% CI -7.3, 11.2; p = 0.68). The two versions of the questionnaire provided similar inferences.

Discussion/conclusion

This trial indicates that, in a healthcare setting, for this complexity of questionnaire (i.e. four attributes and the best-worst scaling design), the use of 16 scenarios obtained very similar response rates to those obtained using half this number.     

Development of a paper-and-pencil semi-adaptive questionnaire for 5 domains of health-related quality of life (PAT-5D-QOL)

Objectives

To develop a paper-and-pencil semi-adaptive test for 5 domains of health-related quality of life (PAT-5D-QOL) based on item response theory (IRT).

Methods

The questionnaire uses items from previously developed item banks for 5 domains: (1) walking, (2) handling objects, (3) daily activities, (4) pain or discomfort, and (5) feelings. For each domain, respondents are initially classified into 4 functional levels. Depending on the level, they are instructed to respond to a different set of 5 additional questions. IRT scores for each domain and overall health utility scores are obtained using a simple spreadsheet. The questions were selected using psychometric and conceptual criteria. The format of the questionnaire was developed through focus groups and cognitive interviews. Feasibility was tested in two population surveys. A simulation study was conducted to compare PAT-5D-QOL with a computerized adaptive test (CAT-5D-QOL) and a fixed questionnaire, developed from the same item banks, in terms of accuracy, bias, precision, and ceiling and floor effects.

Results

Close to 90 % of the participants in feasibility studies followed the skip instructions properly. In a simulation study, scores on PAT-5D-QOL for all domains tended to be more accurate, more precise, less biased, and less affected by a ceiling effect than scores on a fixed IRT-based questionnaire of the same length. PAT-5D-QOL was slightly inferior to a fully adaptive instrument.

Conclusions

PAT-5D-QOL is a novel, semi-adaptive, IRT-based measure of health-related quality of life with a broad range of potential applications.     

A prenotification letter increased initial response,whereas sender did not affect response rates

ObjectiveTo find ways to improve response rates of medical and health surveys. We investigated whether a prenotification letter instead of a second reminder and varying senders of the questionnaires would affect response rates.Study Design and SettingWe present the results of two studies. In the first study, four groups were compared that either received a prenotification letter (group 1 and 2) or a second reminder letter (group 3 and 4); received the questionnaire from either a research institute (group 1 and 3) or a health insurance company (HIC; group 2 and 4). In the second study, we compared two groups that received the questionnaire sent by either a HIC or a hospital. Response rates, response speed, respondent characteristics, item nonresponse, and mean scores on quality aspects and global ratings were compared.ResultsResponse rates did not differ significantly between groups. Prenotification groups returned their questionnaires faster. No other significant differences were found for response speed, respondent characteristics, item nonresponse, or mean scores.ConclusionA prenotification letter does only increase initial response speed and does not increase total response rates. A prenotification letter should be considered when quick response is desirable. Varying senders had no effect on response rates.