Meta-analysis of randomized controlled trials supports the use of incentives for inducing response to electronic health surveys

ObjectivesThe objective of this meta-analysis was to assess incentive effectiveness on response to electronic health surveys.Study Design and SettingA systematic literature search of PubMed, Web of Science, CINAHL, EMBASE, and the Cochrane Library was performed from 1970 to March 2013. Two authors independently selected the trials, assessed methodological quality, and extracted data. Original authors were contacted for the missing information.ResultsThe search strategy yielded nine trials (including 29,463 participants in total) that met eligibility criteria. For each of the incentive strategies examined, a random-effects model was used because of significant heterogeneity, and results were summarized as pooled odds ratios (ORs). Compared with no incentive, the offer of an incentive was seen to have a beneficial effect on response (OR, 1.48; 95% CI: 1.29, 1.71). Specifically, the odds of response were more than doubled when a monetary incentive was used (OR, 2.43; 95% CI: 1.60, 3.69) and increased when nonmonetary incentives were used (OR, 1.33; 95% CI: 1.17, 1.51).ConclusionThese findings indicate that health researchers using electronic surveys can improve the quality of their research by offering incentives to potential participants.     

Enhancing response rates in physician surveys: the limited utility of electronic options

Objective. To evaluate the utility of offering physicians electronic options as alternatives to completing mail questionnaires. Data Source. A survey of colorectal cancer screening practices of Alabama primary care physicians, conducted May–June 2010. Study Design. In the follow‐up to a mail questionnaire, physicians were offered options of completing surveys by telephone, fax, email, or online. Data Collection Method. Detailed records were kept on the timing and mode of completion of surveys. Principal Findings. Eighty‐eight percent of surveys were returned by mail, 10 percent were returned by fax, and only 2 percent were completed online; none were completed by telephone or email. Conclusions. Offering fax options increases response rates, but providing other electronic options does not.     

National health surveys by mail or home interview: effects on response

STUDY OBJECTIVE: To study the effect of using a mail questionnaire or home interviews on the size and the selectivity of response to national health surveys. DESIGN: The interview survey and the mail survey were both carried out in the same country (the Netherlands) using the same sample frame, the same study period (1998) and collected partly the same data on demographic, socioeconomic and health characteristics. SETTING: The Netherlands. PARTICIPANTS: Dutch non-institutionalised inhabitants aged 25 years and over. MAIN RESULTS: Response to the mail survey was lower (46.9%, n=3664) than to the interview survey (58.5%, n=6061). The mail survey gave higher response rates for women and lower response rates for persons with lower levels of education. Respondents to the mail survey reported lower rates of smoking but a slightly worse health status and higher figures on the use of health care services. No differences by method of data collection were found for age, marital status, region, household composition, work status and categories of body mass index. CONCLUSION: Although the response of the mail survey was lower than the home interview survey, respondents showed generally small differences, with exception of level of education.     

Response rates and response bias for 50 surveys of pediatricians

RESEARCH OBJECTIVE: To track response rates across time for surveys of pediatricians, to explore whether response bias is present for these surveys, and to examine whether response bias increases with lower response rates. DATA SOURCE/STUDY SETTING: A total of 63,473 cases were gathered from 50 different surveys of pediatricians conducted by the American Academy of Pediatrics (AAP) since 1994. Thirty-one surveys targeted active U.S. members of the AAP, six targeted pediatric residents, and the remaining 13 targeted AAP-member and nonmember pediatric subspecialists. Information for the full target samples, including nonrespondents, was collected using administrative databases of the AAP and the American Board of Pediatrics. STUDY DESIGN: To assess bias for each survey, age, gender, location, and AAP membership type were compared for respondents and the full target sample. Correlational analyses were conducted to examine whether surveys with lower response rates had increasing levels of response bias. PRINCIPAL FINDINGS: Response rates to the 50 surveys examined declined significantly across survey years (1994-2002). Response rates ranged from 52 to 81 percent with an average of 68 percent. Comparisons between respondents and the full target samples showed the respondent group to be younger, to have more females, and to have less specialty-fellow members. Response bias was not apparent for pediatricians' geographical location. The average response bias, however, was fairly small for all factors: age (0.45 years younger), gender (1.4 percentage points more females), and membership type (1.1 percentage points fewer specialty-fellow members). Gender response bias was found to be inversely associated with survey response rates (r=-0.38). Even for the surveys with the lowest response rates, amount of response bias never exceeded 5 percentage points for gender, 3 years for age, or 3 percent for membership type. CONCLUSIONS: While response biases favoring women, young physicians, and nonspecialty-fellow members were found across the 52-81 percent response rates examined in this study, the amount of bias was minimal for these factors that could be tested. At least for surveys of pediatricians, more attention should be devoted by investigators to assessments of response bias rather than relying on response rates as a proxy of response bias.     

A demonstration of the impact of response bias on the results of patient satisfaction surveys

OBJECTIVES: The purposes of the present study were to examine patient satisfaction survey data for evidence of response bias, and to demonstrate, using simulated data, how response bias may impact interpretation of results. DATA SOURCES: Patient satisfaction ratings of primary care providers (family practitioners and general internists) practicing in the context of a group-model health maintenance organization and simulated data generated to be comparable to the actual data. STUDY DESIGN: Correlational analysis of actual patient satisfaction data, followed by a simulation study where response bias was modeled, with comparison of results from biased and unbiased samples. PRINCIPAL FINDINGS: A positive correlation was found between mean patient satisfaction rating and response rate in the actual patient satisfaction data. Simulation results suggest response bias could lead to overestimation of patient satisfaction overall, with this effect greatest for physicians with the lowest satisfaction scores. CONCLUSIONS: Findings suggest that response bias may significantly impact the results of patient satisfaction surveys, leading to overestimation of the level of satisfaction in the patient population overall. Estimates of satisfaction may be most inflated for providers with the least satisfied patients, thereby threatening the validity of provider-level comparisons.     

Comparing the effectiveness of individualistic,altruistic, and competitive incentives in motivating completion of mental exercises

This study examines the impact of individually oriented, purely altruistic, and a hybrid of competitive and cooperative monetary reward incentives on older adults’ completion of cognitive exercises and cognitive function. We find that all three incentive structures approximately double the number of exercises completed during the six-week active experimental period relative to a no incentive control condition. However, the altruistic and cooperative/competitive incentives led to different patterns of participation, with significantly higher inter-partner correlations in utilization of the software, as well as greater persistence once incentives were removed. Provision of all incentives significantly improved performance on the incentivized exercises. However, results of an independent cognitive testing battery suggest no generalizable gains in cognitive function resulted from the training.     

A new stopping rule for surveys

Non‐response is a problem for most surveys. In the sample design, non‐response is often dealt with by setting a target response rate and inflating the sample size so that the desired number of interviews is reached. The decision to stop data collection is based largely on meeting the target response rate. A recent article by Rao, Glickman, and Glynn (RGG) suggests rules for stopping that are based on the survey data collected for the current set of respondents. Two of their rules compare estimates from fully imputed data where the imputations are based on a subset of early responders to fully imputed data where the imputations are based on the combined set of early and late responders. If these two estimates are different, then late responders are changing the estimate of interest. The present article develops a new rule for when to stop collecting data in a sample survey. The rule attempts to use complete interview data as well as covariates available on non‐responders to determine when the probability that collecting additional data will change the survey estimate is sufficiently low to justify stopping data collection. The rule is compared with that of RGG using simulations and then is implemented using data from a real survey. Copyright © 2010 John Wiley & Sons, Ltd.     

In a randomized study of envelope and ink color, colored ink was found to increase the response rate to a postal questionnaire

OBJECTIVE: To assess the effect of the colors of the envelope and ink on the response rate to a postal questionnaire in a study screening for undiagnosed parkinsonism in people aged 65 years and over in the community. STUDY DESIGN AND SETTING: A total of 2,524 people aged 65 years and over from five general practices in Aberdeen were randomized to receive a questionnaire about the symptoms of parkinsonism printed in either colored (green) or black ink, and sent out in either a brown or white envelope. RESULTS: The overall response rate was 63.5%. There was no significant interaction between envelope and ink color. The use of green ink compared to black significantly increased the response rate from 61.4% to 65.7% (OR 1.20, 95% confidence interval 1.02, 1.41). There was no overall effect of envelope color on response rate (62.3% brown and 64.8% white, OR 0.90, 95% confidence interval 0.76, 1.06) but there was significant heterogeneity between the general practices. When this general practice-envelope interaction was accounted for, brown envelopes had a significantly lower response rate than white ones (OR 0.49). CONCLUSION: This study, along with existing evidence, has shown that the use of certain ink colors in postal questionnaires is likely to increase response rates relative to black ink. The effect of envelope color was inconsistent both within this study and between previous studies.     

A comparison of general practitioner response rates to electronic and postal surveys in the setting of the National STI Prevention Program

Objective: To compare the response rates achieved for an online survey with a postal survey of general practitioners (GPs) as a method to evaluate the National STI Prevention Program. Methods: All GPs in Australia were asked to complete an online survey. A further sample of 509 GPs were asked to complete a postal survey. Response rates to both recruitment methods were compared. The demographic characteristics of responders were compared to the entire GP population of Australia. Results: Twenty GPs completed the online survey (response rate <0.1%). Sixty‐three GPs completed the postal survey (response rate 12.4%). The demographic characteristics of those responding to the postal survey showed no statistically significant difference compared to the general GP population. Conclusion: Our postal survey had a higher response rate than the online survey. Our response to the postal survey was lower than other similar studies and is likely to be due to a lack of incentives and follow‐up. Even with the low response rate it appears that postal surveys can provide a good representation of the overall population. Implications: Despite growing use of online surveys, postal surveys should still be the method of choice whenever possible. Postal surveys should include incentives and further follow‐up of the initial recruitment should be conducted.     

Money as motivation to quit: a survey of a non-random Australian sample of socially disadvantaged smokers' views of the acceptability of cash incentives

Objective

This study aimed to a) assess acceptability of personal financial incentives to socially disadvantaged smokers and non-smokers; b) examine factors associated with acceptability; and c) examine preferred levels of incentive amounts.

Methods

A cross-sectional touch screen computer survey was conducted between February and October 2010 in New South Wales, Australia. Participants were clients experiencing financial or social hardship and receiving emergency welfare aid from a non-government social and community service organisation.

Results

Of 383 participants (69% response rate), 46% believed personal financial incentives were an excellent/good idea, 47% believed personal financial incentives did more good than harm and 61% agreed they would motivate smokers to quit. High acceptability ratings were associated with participants being female, current smokers, living in low socioeconomic areas, experiencing smoking-induced deprivation, making a previous quit attempt and intending to quit in the next 6 months. When asked what amount of incentive they felt would be acceptable, 23% selected amounts between $50 and $500AUD and 37% selected amounts over $500AUD.

Conclusions

Given high smoking prevalence among socially disadvantaged groups and consequent health disparities, it is imperative novel methods of encouraging smoking cessation are explored and tested. This survey found financial incentives may be an acceptable method. Further research to understand all possible positive and negative effects is warranted.     

The effect of question preface on response rates to a telephone survey of sexual experience

Given the heavy reliance on self-report measures in research on sexuality, issues surrounding nonresponse rates are of particular importance. Phone interviews were conducted with 400 adults living in a midsized midwestern city. Rates of nonresponse and reported sexual experience were analyzed as a function of the type of preface statement provided for four questions related to sexual behavior. One preface was general; the other referred to public concern over AIDS. On questions regarding number of sexual partners in the past year and involvement in extramarital sex, women who were given the AIDS preface were significantly more likely to answer the questions than were women who received the general preface. However, for both men and women, reported rates of sexual experience did not differ as a function of the preface statement. Differences between responders and nonresponders were explored. Men were more likely to respond to the sexual items. Education, age, marital status, and religious affiliation were unrelated to rates of response to the sexual questions. For men, responders to the sexuality items reported greater yearly incomes than did nonresponders. Some differences in reported sexual behavior were also found between responders and nonresponders; nonresponders consistently reported less sexual experience. Implications for research on sexuality and future investigation of factors affecting response to sexual items are discussed.     

Lottery incentives did not improve response rate to a mailed survey: a randomized controlled trial

OBJECTIVES: Our study aimed to examine the effect of an instant lottery ticket incentive on the response rate to a mailed questionnaire in a population of trauma patients. STUDY DESIGN AND SETTING: A randomized controlled trial at a major trauma center with 728 patients randomized into 2 groups prior to mailing of a questionnaire. One group of patients (Group A) had a sentence inserted into the cover letter stating that they would receive a $4 instant lottery ticket upon receipt of a completed questionnaire; Group B did not have an incentive. The response rate for both groups was measured after the initial mailing and at the end of the study. The results were analyzed using the X2 test to compare 2 proportions and a P value of .05 was considered significant. RESULTS: The early response rate in Group A was lower than in Group B, and the response rates for both groups were similar at final follow-up. The differences at both time periods were not statistically significant. CONCLUSION: The use of an instant lottery ticket incentive did not improve the response rate to a mailed questionnaire.     

Patterns of response by sociodemographic characteristics and recruitment methods for women in UK population surveys and cohort studies

ABSTRACT

Women are an important public health focus, because they are more likely to experience some social determinants of disease, and they influence family health. Little research has explored the sociodemographic representativeness of women in research studies. We examined the representativeness of female respondents across four sociodemographic factors in UK population surveys and cohort studies. Six UK population-based health surveys (from 2009–2013) and eight Medical Research Council cohort studies (from 1991 to 2014) were included. Percentages of women respondents by age, income/occupation, education status, and ethnicity were compared against contemporary population estimates. Women aged <35 years were under-represented. The oldest women were under-represented in four of nine studies. Within income/occupation, at the highest deprivation level, the range was 4 percent under-representation to 43 percent over-representation; at the lowest level, it was 6 percent under-representation to 21 percent over representation. Of nine studies reporting educational level, four under-represented women without school qualifications, and three under-represented women with degrees. One of five studies over-represented non-white groups and under-represented white women (by 9 percent). Response patterns varied by topic and recruitment and data collection methods. Future research should focus upon the methods used to identify, reach, and engage women to improve representativeness in studies addressing health behaviors.     

'Threats' to and hopes for estimating benefits

In a recent paper in this journal, Andrew Lloyd reviewed some potential threats to the estimation of health care benefits in monetary terms. Particular emphasis was placed on the extent to which the use of non-compensatory heuristics may distort the results. Although it is useful to be reminded of these problems, Lloyd's paper does not do justice to the attention such issues have already received, and continue to receive, within Health Economics. The aim of this paper is twofold. Firstly, it seeks to provide a more balanced and comprehensive view of the evidence by considering some of the methodological work that health economists have conducted in many of the areas pointed out by Lloyd. Secondly, and more importantly, it suggests ways to combine the economic and psychological views of human decision-making, providing a much more positive perspective to all those researchers out there who recognize the necessity of using stated preference methods to inform health policy.     

Using financial incentives to increase fruit and vegetable consumption in the UK

Encouraging the consumption of fruits and vegetables can help people increase their intake of micronutrients, dietary fibre and non‐nutrient substances, which together have benefits for health. A high intake of fruits and vegetables is associated with a lower risk of a number of diseases including heart disease, stroke, type 2 diabetes and some cancers. Among the numerous barriers to purchasing fruits and vegetables, price is still cited as a common factor. Therefore, more needs to be done to incentivise consumers to pick up more of these foods during their weekly shop. The UK Public Health Responsibility Deal seeks to reshape the food environment through behaviour change interventions; however, evidence to support the effectiveness of altering environments to change behaviour as a means to improve population health is weak. In particular, reducing price is a frequently considered policy to improve dietary habits. However, few studies to date have evaluated the effectiveness of such interventions on encouraging fruit and vegetable consumption. Our goal (The Co‐operative Group), in line with the Public Health Responsibility Deal, is to do more to create a positive environment that supports and enables people to increase their fruit and vegetable consumption. In light of this, we piloted a coupon incentive scheme in July 2012 for low fruit and vegetable spenders to see if making these items more affordable would encourage purchases and facilitate increased consumption. Results showed that sales of fruits and vegetables increased by 23.2% during the four‐week campaign, with 132 000 redemptions of the coupon by 90 000 members.     

The Singaporean response to the SARS outbreak: knowledge sufficiency versus public trust

During the outbreak of severe acute respiratory syndrome (SARS) in Singapore from 1 March to 11 May 2003, various national prevention and control measures were undertaken to control and eliminate the transmission of the infection. During the initial period of the epidemic, public communication was effected through press releases and media coverage of the epidemic. About a month into the epidemic, a public education campaign was mounted to educate Singaporeans on SARS and adoption of appropriate behaviours to prevent the spread of the disease. A survey was conducted in late April 2003 to assess Singaporeans' knowledge about SARS and infection control measures, and their concerns and anxiety in relation to the outbreak. The survey also sought to assess their confidence in the ability of various institutions to deal with SARS and their opinion on the seemingly tough measures enforced. The study involved 853 adults selected from a telephone-sampling frame. Stratified sampling was used to ensure adequate representation from major ethnic groups and age groups. The study showed that the overall knowledge about SARS and control measures undertaken was low (mean per cent score of 24.5 +/- 8.9%). While 82% of respondents expressed confidence in measures undertaken by Tan Tock Seng Hospital (the hospital designated to manage SARS), only 36% had confidence in nursing homes. However, >80% of the public agreed that the preventive and control measures instituted were appropriate. Despite the low knowledge score, the overall mean satisfaction score of the government's response to SARS was 4.47 (out of possible highest score of 5.00), with >93% of adult Singaporeans indicating that they were satisfied or very satisfied with the government's response to SARS. Generally, Singaporeans had a high level of public trust (satisfaction with government, confidence in institutions, deeming government measures appropriate), scoring 11.4 out of possible maximum of 14. The disparity between low knowledge on the one hand and high confidence and trust in the actions of the government on the other suggests that Singaporeans do not require high knowledge sufficiency to be confident in measures undertaken by the government to control the SARS crisis.