A comparison of responses to mailed questionnaires and telephone interviews in a mixed mode health survey

This paper compares respondents to mailed questionnaires with those nonrespondents subsequently interviewed by telephone in a survey of Massachusetts women aged 45-55 years conducted in 1981-1982. This mixed mode approach produced 8,050 responses, giving a response rate of 77%. This rate is similar to rates obtained in many surveys that employed in-person interviews, which are still widely used in health surveys but are increasingly expensive. Telephone respondents differed socioeconomically from mail respondents, suggesting that telephone follow-up of nonrespondents may have reduced nonresponse bias in this survey. Thus, a mixed mode approach may be superior to a mail-only approach with respect to this aspect of data quality. Women responding by mail were more likely to hold professional jobs, to have relatively high household incomes, and to have more years of education. Controlling for these socioeconomic differences did not, however, remove differences in reported health outcomes between mail and telephone respondents. These differences may be explained by less complete recall in the telephone interviews or they may arise from actual differences in health profiles between early (i.e., mail) and late (i.e., telephone) respondents. Although a mixed mode approach may reduce nonresponse bias, more research is required concerning the reasons for response differences between modes and to eliminate any differences caused by problems in data quality.     

The effect of a monetary incentive in increasing the return rate of a survey to family physicians

The barrage of requests family physicians receive to complete mail surveys often results in physicians who are unwilling, or unable due to time constraints, to complete each survey they receive. Thus, to obtain an acceptable response rate, state-of-the-art mail survey techniques must be used. This article reports the results of the use of a modest ($1) monetary incentive to increase a survey response rate. A random sample of 600 American Academy of Family Physicians members were mailed a survey of firearm safety counseling; half received a $1 incentive whereas the remaining half served as a control group. The response rate in the incentive group was 63% compared to 45% in the control group [chi 2 (1, N = 251) = 16.0, p < .001]. Further, the use of the incentive appears to be more cost-effective than a third follow-up (postcard reminder) mailing.     

Cost-effectiveness of a lottery for increasing physicians' responses to a mail survey

To evaluate the cost-effectiveness of a lottery on physicians' responses to a mail survey, a randomized controlled trial was conducted with a random sample of 1,000 members of the Quebec Federation of General Practitioners in 1997. For the first mailing of this survey, each respondent was randomly assigned to the control or experimental group, which was offered participation in a lottery upon return of the questionnaire. Response rate was 41.2% in the experimental group and 34.8% in the control group, a 6.4% difference (CI95%: 0.6%-12.6%). The additional cost of the lottery was about Can$500, giving an incremental cost of Can$16 per questionnaire returned. In conclusion, a lottery resulted in a small but statistically significant increase in the response rate of physicians to a mail survey. This method may be a cost-effective option when applied to large surveys.     

The effect on response rates of offering a small incentive with a mailed questionnaire.

To determine whether response rates to a mailed questionnaire sent to population control subjects could be increased through offer of a small incentive, half of the control subjects (n = 477) in a case-control study of renal cell carcinoma were randomly selected to receive a contact letter offering a lottery ticket if a completed questionnaire was returned; the remaining subjects (n = 477) received the same letter but with no mention of a lottery ticket. Overall response rates did not differ between the two groups (72.6% versus 74.4%), although a higher percentage of those offered a lottery ticket responded without follow-up (24.4% versus 18.5%). Binomial regression modeling of the effect of the lottery ticket offer, sex, age, and percent of urban dwellers on response indicated a significant effect only for percent of urban dwellers, the rate of response increasing with a decreasing percentage of urban dwellers. The effect of sex was of borderline significance (P = 0.05), with females having the higher rate of response.     

Medical consumption of respondents and non-respondents to a mailed health survey

Non-response bias can distort the results of health surveys.The occurrence of selective non-response can be assessed whendata are available for both respondents and non-respondents.The objective of this study was to compare the medical consumptionof respondents and non-respondents to a mailed health survey.A mailed health survey was conducted among approximately 13,500adults and among parents of approximately 1,500 children aged5–15 years. The net response rate was 70.4%. A panel dataset that could be matched with the health survey data was availablefor all eligible persons. This data set comprises administrativeinformation on hospitalizations, annual health care expendituresand demographic variables. The results of this study show thatresponse was associated with age, sex, degree of urbanizationand type of insurance. After correcting for differences in demographicvariables, respondents and non-respondents differ in the utilizationof several types of care. Relatively more users than non-usersresponded. Response was not associated with the utilizationof care related to severe conditions such as in-patient hospitalcare. The conclusion from this study is that when a mailed healthsurvey is used to measure medical consumption, the non-responsebias will result in a small overestimation of utilization.     

Assessing the impact of payment method and practice setting on German physicians' practice patterns

This project determined the impact that a physician's practice setting and reimbursement method has on his or her practice behavior. Multivariate regressions that controlled for physician, patient, and practice characteristics were conducted. The primary data source was a questionnaire that sampled ambulatory physicians practicing in the state of Brandenburg, Germany. This research demonstrated that physicians paid on a fee-for-service basis differ significantly from practitioners paid a salary in captured utilization measures: more patient visits per week, including more follow-up visits; a decreased rate of hospitalization; and an increased likelihood of making house calls. A group practice setting demonstrated little impact when compared with a solo practice.     

Improving response rates through incentive and follow-up: the effect on a survey of physicians' knowledge of genetics.

OBJECTIVES. This study assessed efforts to increase response rates to a mailed physician survey and examined whether, as a result, nonresponse bias was reduced. METHODS. Randomly selected physicians and geneticists were mailed a questionnaire concerning genetics knowledge and attitudes. In the final but not the pilot survey, a $25 incentive and intensive follow-up were used to increase the response rate. RESULTS. The response rate from physicians in the final survey was 64.8% (n = 1140), compared with 19.6% in the pilot test (n = 69). Sample representatives in sociodemographic and practice characteristics was improved by follow-up. Respondents recruited with more difficulty did not differ on the principal outcome variable, genetics knowledge, except on one subscore. Pilot study and final survey respondents did not differ in knowledge. CONCLUSIONS. Although the effect of increased response rates on the principal outcome variable in this study was minimal, this may not be the case for other studies. Every effort should be made to attain as high a response rate as is practical and to establish that respondents are representative of the population being sampled.     

Stress management in primary care: physicians' beliefs and patterns of practice

We examined attitudes and practices related to stress management in a random sample of Texas primary care physicians. Two fifths of the physicians indicated that avoiding undue stress was very important, and more than one third reported that they gathered information about stress on a routine basis. Over two thirds of the physicians said that they had counseled their patients regarding stress and 13 percent had referred patients with high stress to outside programs. The physicians' confidence that they had the skills to help patients with high stress, their belief in the benefits of avoiding undue stress, and their estimate of patient follow-through on recommendations were positively associated with history-taking and counseling regarding stress. Recommendations for medical education include incorporating information about stress and health into medical school curriculums, building skills in stress management, and increasing feedback from patients who have coped successfully with high stress.     

The influence of rural physicians' perception on referral patterns to a university hospital.

The patient referral process is based on a complex set of social rewards and costs for the referring physician. For the physician or institution potentially receiving referrals, a key question is, why is one institution selected over other available choices? Factor analysis of a survey of rural physicians revealed five factors of reward and cost associated with the choice of where to refer patients. Further analysis of these factors suggested that the ability to have patients admitted and treated when necessary is central to the satisfaction of the referring physician and their willingness to continue the consultant relationship. Adequate information about the consultants was also important to satisfaction. Other factors have less influence. Further, “admissions when necessary” was the only reward/cost factor which predicted physicians who referred more to the university hospital than to other hospitals. If a university hospital wants to develop strategies to encourage referrals from rural physicians, it must be cognizant of these social reward-cost factors.     

The effects of continuing medical education credits on physician response rates to a mailed questionnaire

This study investigated whether the opportunity to obtain Continuing Medical Education (CME) credit together with a five-dollar bill increased response rates and questionnaire completion rates in a physician survey involving mailed questionnaires. One thousand, three hundred and fourteen cardiologists, family practitioners, general internists (non-surgeons) and 264 vascular surgeons randomly identified from the American Medical Association database participated. After two, of up to four, questionnaire mailings, the opportunity to obtain CME credit and a five-dollar bill were included with questionnaire mailings. Among non-surgeons, 26.5% responded to pre-incentive mailings and 30.2% of those initially unresponsive replied after the interventions. Among surgeons, 39% responded to pre-incentive mailings and 32.7% of those initially unresponsive replied after the interventions. In conclusion, the opportunity to receive CME credit combined with a small monetary incentive is an effective motivation for physicians participating in a study involving mailed questionnaires.     

The effects of cash and lottery incentives on mailed surveys to physicians: a randomized trial

Low response rates, especially among physicians, are a common problem in mailed survey research. We conducted a randomized trial to examine the effects of cash and lottery incentives on response rates. A total of 4,850 subjects were randomized to one of three interventions accompanying a mailed survey-no incentive (n = 1,700), cash payment [three levels of Hong Kong dollars (HKD) $10, $20, and $40; N = 50 in each subgroup], or entry into a lottery (three levels of HKD$1,000, $2,000, and $4,000; N = 1,000 in each subgroup) on receipt of the completed questionnaire. The response rates were higher among those offered incentives than those without (19.8% vs. 16.8%, P =.012). Cash was the more effective incentive compared to lottery (27.3% vs. 19.4%, P =.017). Response also increased substantially between the first and second mailings (14.2% vs. 18.8%, P >.001). In addition, those with specialist qualifications were more willing to participate in mailed surveys. We found no significant differences in response outcomes among the various incentive arms. Cash reward at the $20 level was the most cost-effective intervention, in terms of cost per responder. Further systematic examination of the effects of different incentive strategies in epidemiologic studies should be encouraged.     

Randomized trial of financial incentives and delivery methods for improving response to a mailed questionnaire

In a follow-up study, only 64% of 126,628 US radiologic technologists completed a questionnaire during 1994-1997 after two mailings. The authors conducted a randomized trial of financial incentives and delivery methods to identify the least costly approach for increasing overall participation. They randomly selected nine samples of 300 nonresponders each to receive combinations of no, 1.00 US dollar, 2.00 US dollars, and 5.00 US dollars cash or check incentives delivered by first-class mail or Federal Express. Federal Express delivery did not achieve greater participation than first-class mail (23.2% vs. 23.7%). In analyses pooled across delivery methods, the response was significantly greater for the 2.00 US dollar bill (28.9%, 95% confidence interval (CI): 25.2, 32.7; p < 0.0001), 5.00 US dollars check (27.5%, 95% CI: 22.5, 33.0; p = 0.0001), 1.00 US dollar bill (24.6%, 95% CI: 21.2, 28.3; p = 0.0007), and 2.00 US dollars check (21.8%, 95% CI: 18.5, 25.3; p = 0.02) compared with no incentive (16.6%, 95% CI: 13.7, 19.9). The response increased significantly with increasing incentive amounts from 0.00 to 2.00 US dollars cash (p trend < 0.0001). The 2.00 US dollar bill achieved a 30% greater response than did a 2.00 US dollars check (p = 0.005). For incentives sent by first-class mail, the 5.00 US dollars check yielded 30% greater participation than did the 2.00 US dollars check (p = 0.07). A 1.00 US dollar bill, chosen instead of the 2.00 US dollars bill because of substantially lower overall cost and sent by first-class mail to the remaining 42,717 nonresponders, increased response from 64% to 72%.     

Prevalence and patterns of youth responses to standard disability survey questions

BackgroundIn 2011, the US Department of Health and Human Services adopted a minimum set of six standardized questions about disability to be used in population-based health surveys. These questions have been validated for self- and proxy-report use by adults, but how they perform for adolescents is unknown.ObjectiveTo describe how 8th grade students, 11th grade students, and young adults aged 18–24 years in Oregon answer these questions.MethodsCross-sectional study design. Data for the 8th and 11th grade students were derived from the Oregon Health Teens survey (OHT; 2017 and 2019); data for young adults aged 18–24 were from the Behavioral Risk Factor Surveillance System (BRFSS; 2017 and 2018). Unweighted counts, weighted proportions and 95% confidence intervals were calculated for socio-demographic characteristics, the six disability questions, and overall disability status (yes/no) among 8th graders (n = 14,396), 11th graders (n = 23,517), and young adults (n = 1112).ResultsResponses for 8th and 11th grade students were materially consistent for all six questions. Young adults were markedly less likely to report cognitive disability compared to 8th and 11th graders (17.2% vs. 24.9% and 27.0%, respectively) and somewhat less likely to report an independent living disability (6.5% vs. 8.6% and 9.8%, respectively).ConclusionDifferences in cognitive disabilities between adolescents and young adults may either be due to differences in underlying impairment or the result of youth interpreting this question differently than adults. Validation of the standardized disability identifiers for self-report in adolescents is needed.     

Evaluation of the impact of a low back pain educational intervention on physicians' practice patterns and patients' outcomes

This study was designed to evaluate the effects of an educational intervention on physicians' management of low back pain patients. The study universe composed of 64 providers employed by a large occupational health group, with 4411 LBP cases pretraining and 4665 cases posttraining selected from the organization's database. The control group had 151 providers with 8478 pretraining and 8876 posttraining cases. Results showed that practices of physicians who participated in the intervention strategy underwent significant changes between the two intervals. Specifically, the intervention group reduced the percentage of restricted work cases, reduced the percentage of lost-time cases for male patients and female patients (less than 40 years old), and shortened restricted workday duration and total case duration for female patients. These results provide early indication for the effectiveness of this type of educational intervention strategy.     

The effects of lottery incentive and length of questionnaire on health survey response rates: a randomized study.

Maximizing the response rate of self-administered questionnaires is key in survey research. We aimed to evaluate the effects of lottery incentive and length of questionnaire on health survey response rates when used in isolation or combined. A random sample of 440 residents in Western Sydney, Australia was randomly allocated to four equal groups to receive or not receive an instant lottery ticket and a long (seven page) or short (one page) questionnaire. The overall response rate was 71.8%. The final response rates were higher among those receiving the short, rather than the long, questionnaire (75.6% versus 68.2%) (P = 0.08); and among those receiving the lottery incentive compared with those not receiving the incentive (75% versus 68.2%) (P = 0.09). By logistic regression analysis, the success of obtaining a completed questionnaire without any follow-up reminders was significantly associated with the lottery incentive but not the questionnaire length (P = 0.03 and P = 0.54, respectively). The difference between lottery and no lottery groups decreased gradually during the follow-up. A lottery incentive is associated with an increased response after the first mailing. A small up-front cost for a lottery ticket may be worthwhile, since it can save further costs by obviating the need for repeated follow-ups.     

The effects of mailed reminders and tailored messages on mammography screening

A mail campaign to promote mammography screening was tested with 3,887 Medicare recipients in North Dakota who had not had a mammogram in 2 years. Three types of mailings were compared: (1) a simple reminder message, (2) a reminder accompanied by a persuasive communication emphasizing personal risk, and (3) a reminder accompanied by a message tailored to the participants' chief barrier to having a mammogram. Overall, subsequent mammography rates for women in these conditions did not differ from the rate observed among women who did not receive any mailing. However, post-hoc analyses suggested that women who reported a barrier to having a mammogram were more likely to have a mammogram. Population-wide mail campaigns of the kinds tested here may be generally ineffective for Medicare recipients who are obtaining screenings infrequently. Tailoring messages may be one potentially effective intervention, if investigators can develop ways to increase responses to inquiries about barriers.     

The experiences of districts in implementing a national incentive programme to promote safe delivery in Nepal

Background  

Nepal's Safe Delivery Incentive Programme (SDIP) was introduced nationwide in 2005 with the intention of increasing utilisation of professional care at childbirth. It provided cash to women giving birth in a health facility and an incentive to the health provider for each delivery attended, either at home or in the facility. We explored early implementation of the programme at the district-level to understand the factors that have contributed to its low uptake.     

The experiences of districts in implementing a national incentive programme to promote safe delivery in Nepal

Background

Providing timely access to physiotherapy has long been a problem for the National Health Service in the United Kingdom. In an attempt to improve access some physiotherapy services have introduced a new treatment pathway known as PhysioDirect. Physiotherapists offer initial assessment and advice by telephone, supported by computerised algorithms, and patients are sent written self-management and exercise advice by post. They are invited for face-to-face treatment only when necessary. Although several such services have been developed, there is no robust evidence regarding clinical and cost-effectiveness, nor the acceptability of PhysioDirect.

Methods/Design

This protocol describes a multi-centre pragmatic individually randomised trial, with nested qualitative research. The aim is to determine the effectiveness, cost-effectiveness, and acceptability of PhysioDirect compared with usual models of physiotherapy based on patients going onto a waiting list and receiving face-to-face care. PhysioDirect services will be established in four areas in England. Adult patients in these areas with musculoskeletal problems who refer themselves or are referred by a primary care practitioner for physiotherapy will be invited to participate in the trial. About 1875 consenting patients will be randomised in a 2:1 ratio to PhysioDirect or usual care. Data about outcome measures will be collected at baseline and 6 weeks and 6 months after randomisation. The primary outcome is clinical improvement at 6 months; secondary outcomes include cost, waiting times, time lost from work and usual activities, patient satisfaction and preference. The impact of PhysioDirect on patients in different age-groups and with different conditions will also be examined. Incremental cost-effectiveness will be assessed in terms of quality adjusted life years in relation to cost. Qualitative methods will be used to explore factors associated with the success or failure of the service, the acceptability of PhysioDirect to patients and staff, and ways in which the service could be improved.

Discussion

It is still relatively unusual to evaluate new forms of service delivery using randomised controlled trials. By combining rigorous trial methods with economic analysis of cost-effectiveness and qualitative research this study will provide robust evidence to inform decisions about the widespread introduction of PhysioDirect services.

Trial registration

Current Controlled Trials ISRCTN55666618