Applying mail response enhancement techniques to health care surveys: a cost-benefit approach

A study of over 1200 persons gave insights into the effectiveness of different appeals in a mail survey for a health-care facility. The influences of financial incentives, contributions to charity and time appeals were studied. Not only was the practical effectiveness of the three different appeals investigated, the cost of obtaining the response rates with each was also shown. The range of response rates was from 25% to 67%. The average cost per return for the different methods ranged from $2.87 to $5.64. A method of testing for the potential bias in a survey is also presented.     

The use of monetary incentives in a community survey: impact on response rates, data quality, and cost

OBJECTIVES: To assess the effect of incentive size on response rates, data quality, and cost in a digestive health status mail survey of a community sample of health plan enrollees. DATA SOURCES/SETTING: The study population was selected from a database of enrollees in various health plans obligated to receive care at Park Nicollet Clinic-HealthSystem Minnesota, a large, multispecialty group in Minneapolis, Minnesota, and the nearby suburbs. STUDY DESIGN: A total of 1,800 HealthSystem Minnesota enrollees were randomly assigned to receive a survey with an incentive of $5 or $2. The response rates for each incentive level were determined. Data quality, as indicated by item nonresponse and scale scores, was measured. Total cost and cost per completed survey were calculated. PRINCIPAL FINDINGS: The response rate among enrollees receiving $5 (74.3 percent) was significantly higher than among those receiving $2 (67.4 percent); differences were more pronounced in the first wave of data collection. Data quality did not differ between the two incentive groups. The total cost per completed survey was higher in the $5 condition than in the $2 condition. CONCLUSIONS: A $5 incentive resulted in a higher response rate among a community patient sample with one mailing than did a $2 incentive. However, the response rates in the $2 condition approached the level of the $5 incentive, and costs were significantly lower when the full follow-up protocol was completed. Response rates were marginally increased by follow-up phone calls. The incentive level did not influence data quality. The results suggest if a survey budget is limited and a timeline is not critical, a $2 incentive provides an affordable means of increasing participation.     

Effectiveness of monetary incentives for recruiting adolescents to an intervention trial to reduce smoking

BACKGROUND: The study objective is to evaluate the effect of monetary incentives on response rates of adolescents to a smoking-related survey as the first step toward participation in an intervention trial. METHODS: A sample of 4,200 adolescent members of a managed care organization were randomized to one of four incentive groups: a $2 cash group, a $15 cash group, a $200 prize drawing group, or a no-incentive group. We compared group-specific response rates and willingness to be contacted about future study activities, as well as costs. RESULTS: Incentives increased survey response rates (55% response without incentive vs. a 69% response with incentive), with response of 74% in the $15 cash group, 69% in the token group, and 63% with a prize incentive. Incentives did not adversely affect willingness of adolescents to be contacted about a smoking intervention, (65% willing with incentives vs. 60% without, P = 0.03). In terms of cost per additional survey completed, token and prize groups were marginally more expensive than the no-incentive group ($0.40 and $1.42, respectively) while the large cash incentive was substantially more costly ($11.37). CONCLUSIONS: Monetary incentives improve response rates to a mailed survey, without adverse impact on willingness to further participate in intervention activities. However, a variety of issues must be considered when using incentives for recruitment to intervention studies.     

The effects of cash and lottery incentives on mailed surveys to physicians: a randomized trial

Low response rates, especially among physicians, are a common problem in mailed survey research. We conducted a randomized trial to examine the effects of cash and lottery incentives on response rates. A total of 4,850 subjects were randomized to one of three interventions accompanying a mailed survey-no incentive (n = 1,700), cash payment [three levels of Hong Kong dollars (HKD) $10, $20, and $40; N = 50 in each subgroup], or entry into a lottery (three levels of HKD$1,000, $2,000, and $4,000; N = 1,000 in each subgroup) on receipt of the completed questionnaire. The response rates were higher among those offered incentives than those without (19.8% vs. 16.8%, P =.012). Cash was the more effective incentive compared to lottery (27.3% vs. 19.4%, P =.017). Response also increased substantially between the first and second mailings (14.2% vs. 18.8%, P >.001). In addition, those with specialist qualifications were more willing to participate in mailed surveys. We found no significant differences in response outcomes among the various incentive arms. Cash reward at the $20 level was the most cost-effective intervention, in terms of cost per responder. Further systematic examination of the effects of different incentive strategies in epidemiologic studies should be encouraged.     

Increasing response rates for mailed surveys of Medicaid clients and other low-income populations.

Mailing surveys to low-income populations is often avoided because of concern about low response rates. In this study, the authors used a mailed survey of a low-income population to test whether $1.00 or $2.00 cash-response incentives were worth the expense and whether 2-day priority mail ($2.90 postage) would yield a sufficiently higher response rate than certified mail ($1.52 postage) to justify its cost. In 1994, 2,243 randomly selected families in subsidized health care programs in Pierce County, Washington, were randomly sent no incentive, $1.00, or $2.00 in the first of three mailings. For the third mailing, nonrespondents were randomly assigned to receive either certified or 2-day priority mail. After 4 weeks, the response rates were 36.7%, 48.1%, and 50.3% for the no-incentive, $1.00, and $2.00 groups, respectively. After three mailings, the cost per response was the lowest for the group that received $1.00. The response rate for the certified mailing (28.1%) was significantly higher than the rate for the more expensive priority mailing (21.7%). No incentive-related bias was detected. The authors concluded that the most efficient protocol for this low-income population was to use a $1.00 incentive in the first mailing and a certified third mailing.     

Use of Monetary and Nonmonetary Incentives to Increase Response Rates Among African Americans in the Wisconsin Pregnancy Risk Assessment Monitoring System

From 2009 to 2010, an experiment was conducted to increase response rates among African American mothers in the Wisconsin Pregnancy Risk Assessment Monitoring System (PRAMS). Sample members were randomly assigned to groups that received a prepaid, cash incentive of $5 (n = 219); a coupon for diapers valued at $5 (n = 219); a coupon for diapers valued at 6 (n = 210); or no incentive (n = 209). Incentives were included with the questionnaire, which was mailed to respondents. We examined the effects of the incentives on several outcomes, including response rates, cost effectiveness, survey response distributions, and item nonresponse. Response rates were significantly higher for the cash group than for the coupon (42.5 vs. 32.4%, P < .05) or no incentive group (42.5 vs. 30.1%, P < .01); the coupon and no incentive groups performed similarly. While absolute costs were the highest for the cash group, the cost per completed survey was the lowest. The incentives had limited effects on response distributions for specific survey questions. Although respondents completing the survey by mail in the cash and coupon groups exhibited a trend toward being less likely to have missing data, the effect was not significant. Compared to a coupon or no incentive, a small cash incentive significantly improved response rates and was cost effective among African American respondents in Wisconsin PRAMS. Incentives had only limited effects, however, on survey response distributions, and no significant effects on item nonresponse.     

Cost-effectiveness of routine and campaign vaccination strategies in Ecuador

A national household coverage survey of 3697 Ecuadorean children, carried out in July 1986, provided an opportunity for a cost-effectiveness analysis of (1) routine vaccination services based in fixed facilities and (2) mass immunization campaigns. A major purpose of the campaigns was to complement the routine services and to accelerate immunization activities. Based on the coverage survey, the Program for Reduction of Maternal and Childhood Illness (PREMI) and earlier campaigns increased the proportion of children under 5 years who were fully vaccinated from 43% to 64%. In one year, the PREMI campaign was responsible for fully vaccinating 11% of children under one year, 21% of 1-2-year-old children, and 13% of all children under 5 years. The campaign also helped ensure that vaccinations were completed when children were still very young and at greatest risk. The average cost per vaccination dose (in 1985 US$ prices) was approximately $0.29 for fixed facilities and $0.83 for the PREMI campaign. Total national costs were $675,000 and $1,665,000 for routine and campaign services respectively. The cost per fully vaccinated child (FVC) was $4.39 for routine vaccination services and $8.60 for the campaign. The cost per death averted was about $1900 for routine vaccination services, $4200 for the PREMI campaign, and $3200 for the combined programme. Because of Ecuador's lower mortality rates, the costs per death averted in Ecuador from both vaccination strategies are not as low as those from studies of vaccinations in Africa. The campaigns, though less cost-effective than routine services, significantly improved the vaccination coverage of younger children who had been missed by the routine services. The costs per FVC of both the campaign and the routine services compare favourably with such programmes in other countries.     

Optimal design features for surveying low-income populations

Improving Medicaid program effectiveness for underserved populations is hampered by low survey response rates. This study determined how to maximize Medicaid consumer satisfaction survey response rates to the Consumer Assessment of Health Plans Study (CAHPS) survey. In a public immunization clinic, 8 focus groups and 15 extended interviews were used to assess consumer-preferred survey design features and incentives. To test hypotheses, we conducted the following trial. Out of 10,733 total participants in a Kansas Medicaid managed care plan, 3,685 eligible for CAHPS were unduplicated by household. After randomization of the 968 households with valid addresses to one of three groups, a controlled trial was conducted to assess response rates to CAHPS survey formats and incentives. Response rates were 35% for a standard mailing, 44% for a user-friendly low-literacy mailing, and 64% for a user-friendly low-literacy mailing with a $10 contingent incentive. Both experimental arms significantly improved response compared with the control; the response rate of the mailing group with the incentive was higher than the response rate of the group receiving that mailing without any incentive (p<0.0001). Using consumer-based preferences significantly increased response rates to this Medicaid satisfaction survey. Raising CAHPS response rates may increase validity of Medicaid consumer satisfaction information.     

Differential effectiveness of telephone prompts by medical and nonmedical staff in increasing survey response rates: a randomised trial

Abstract: We conducted a randomised trial to determine the differential effectiveness of a telephone prompt by a medical researcher compared with a nonmedical research assistant in improving response rates of general practitioners to a survey and to compare personnel costs. A national random sample of Australian general practitioners was allocated randomly to two intervention groups. In advance of a self–administered questionnaire, Group A (n = 184) received a telephone prompt by a medical researcher and Group B (n = 189) a prompt by an experienced nonmedical research assistant. Other aspects of survey administration were identical for both groups. The five–month cumulative response rate obtained by the medical researcher (81 per cent) was not significantly different from that of 72 per cent obtained by the research assistant ( X ²= 3.3, 1 df, P = 0.07). For Group A, 279 telephone calls, consuming 23 hours 15 minutes, were made. Group B required more calls (384) and more time (32 hours). Using the relevant award pay scales, the estimated personnel costs for each group were $631 and $601 respectively. We conclude that an experienced nonmedical research assistant is as effective as a medical practitioner in administering telephone prompts to enhance survey response rates, although savings are not necessarily made.     

Cost effectiveness of community-based therapeutic care for children with severe acute malnutrition in Zambia: decision tree model

Background

Children aged under five years with severe acute malnutrition (SAM) in Africa and Asia have high mortality rates without effective treatment. Primary care-based treatment of SAM can have good outcomes but its cost effectiveness is largely unknown.

Method

This study estimated the cost effectiveness of community-based therapeutic care (CTC) for children with severe acute malnutrition in government primary health care centres in Lusaka, Zambia, compared to no care. A decision tree model compared the costs (in year 2008 international dollars) and outcomes of CTC to a hypothetical 'do-nothing' alternative. The primary outcomes were mortality within one year, and disability adjusted life years (DALYs) after surviving one year. Outcomes and health service costs of CTC were obtained from the CTC programme, local health services and World Health Organization (WHO) estimates of unit costs. Outcomes of doing nothing were estimated from published African cohort studies. Probabilistic and deterministic sensitivity analyses were done.

Results

The mean cost of CTC per child was $203 (95% confidence interval (CI) $139–$274), of which ready to use therapeutic food (RUTF) cost 36%, health centre visits cost 13%, hospital admissions cost 17% and technical support while establishing the programme cost 34%. Expected death rates within one year of presentation were 9.2% with CTC and 20.8% with no treatment (risk difference 11.5% (95% CI 0.4–23.0%). CTC cost $1760 (95% CI $592–$10142) per life saved and $ 53 (95% CI $18–$306) per DALY gained. CTC was at least 80% likely to be cost effective if society was willing to pay at least $88 per DALY gained. Analyses were most sensitive to assumptions about mortality rates with no treatment, weeks of CTC per child and costs of purchasing RUTF.

Conclusion

CTC is relatively cost effective compared to other priority health care interventions in developing countries, for a wide range of assumptions.     

A cost-effectiveness analysis of hepatitis B vaccine in predialysis patients.

OBJECTIVE. Our objective was to assess the cost effectiveness of hepatitis B vaccine in predialysis patients. DATA SOURCES. Costs were calculated from estimated rates of health services use and unit costs of resource use. Efficacy data were based on probability estimates from the medical literature and included vaccination response rates, anticipated hepatitis B virus (HBV) infection rates, and outcomes from HBV. STUDY DESIGN. Costs and effectiveness of HBV vaccination was modeled with a decision tree constructed to analyze three vaccination strategies for patients with renal insufficiency: vaccine given prior to dialysis, vaccine given at time of dialysis, and no vaccine. Sensitivity analyses were performed to assess the effect of varying important clinical and cost variables. DATA COLLECTION/EXTRACTION METHODS. All analyses were based on efficacy and cost estimates derived from the medical literature. Analyses were conducted with the aid of SMLTREE software. PRINCIPAL FINDINGS. The number of patients requiring vaccination per case of HBV prevented was higher for dialysis patients (625 vaccinees/case prevented) than for predialysis patients (434 vaccinees/case prevented). The cost-effectiveness ratios were $25,313/case of HBV prevented for vaccination at the time of dialysis and $31,111 for the predialysis vaccine. When a higher HBV infection rate (based on clinical trial data) was substituted in the analysis, the cost effectiveness of a predialysis vaccination strategy improved to $856 per case prevented. Results were sensitive to the cost of the vaccine and the incidence of HBV infection in dialysis patients. For the predialysis strategy to become cost saving, the price of the vaccine would have to decrease from $114 to $1.50, or the incidence of infection would have to increase from 0.6 percent to 38 percent, holding all other variables constant. CONCLUSIONS. Additional HBV infection can be prevented by immunizing predialysis patients, but the cost is high. Decisions concerning vaccination policy should be influenced by local prevalence of HBV infection.     

Cost-effectiveness of follow-up contact for a postal survey: a randomised controlled trial

Objective: This study examines the effectiveness and costs of follow‐up phone calls in improving response rates to a community survey. Methods: Non‐responders to a postal survey were randomly allocated to receive a phone call or no phone call. The resources used for the development and implementation of the survey were documented. The response rates and cost per level of follow‐up contact examined. Results: Follow‐up phone calls led to a statistical significant increase in the number of responses to a community‐wide survey, relative to no phone call. This relative increase in responses (n=62 for the follow‐up phone call group versus n=1 for controls), did not increase the absolute survey response rate sufficiently (from 38.5% for two mailed surveys to 39.8% for two mailed surveys plus a phone call) to justify the phone call costs. Scenario analyses show increasing the initial response rate by 10% and conducting a second mailed survey achieves greater marginal cost savings than increasing the response rate to the second mailout or the follow‐up phone calls. Conclusions: These results suggest a follow‐up phone call was not cost effective. Survey research ought to primarily focus on obtaining optimal initial response rates by using strategies identified in a Cochrane meta‐analytic review.     

Response rates to random digit dialing for recruiting participants to an onsite health study.

OBJECTIVE: To evaluate the response rates when random digit dialing was used as a substitute for geographic area sampling and household interviews to recruit 2100 African Americans for a blood pressure measurement and hypertension-related knowledge and attitudes survey. METHODS: Random digit dialing was used to identify African American adults living in 12 low-income ZIP code areas of Houston, Texas. A brief survey of hypertension awareness and treatment was administered to all respondents. Those who self-identified as African American were invited to a community location for blood pressure measurement and an extended personal interview. An incentive of $10 was offered for the completed clinic visit. A substudy of nonrespondents was carried out to test the effectiveness of a $25 incentive in increasing the response rate. Data from the initial random telephone interview were used to identify differences between those who did and did not attend the measurement session. RESULTS: Ninety-four percent of eligible persons contacted completed the telephone survey, and 65% agreed to visit a central community site for blood pressure measurement. In spite of the financial incentive and multiple attempts to reschedule missed appointments, only 26% of the 65% who agreed to attend completed the scheduled visit. In the substudy of the higher financial incentive, all of those who missed the original appointment agreed to another appointment, and 85% of this subgroup kept it. Not being employed full-time and a history of hypertension were consistently associated with agreement to be measured and keeping an appointment. In spite of the low response rate for scheduled appointments, differences--other than in employment status and a history of hypertension--between responders and nonresponders were small and consistent with what is usually observed in health surveys. CONCLUSIONS: The use of random digit dialing as a substitute for area sampling and household screening resulted in unacceptably low response rates in the study population and should not be undertaken without further research on ways to increase response rates.     

The cost-effectiveness of cash versus lottery incentives for a web-based, stated-preference community survey

We present the results of a randomized experiment to test the effectiveness and cost-effectiveness of response incentives for a stated-preference survey of a general community population. The survey was administered using a mixed-mode approach, in which community members were invited to participate using a traditional mailed letter using contact information for a representative sample of the community; but individuals completed the survey via the web, which exploited the advantages of electronic capture. Individuals were randomized to four incentive groups: (a) no incentive, (b) prepaid cash incentive ($2), (c) a low lottery (10 prizes of $25) and (d) a high lottery (2 prizes of $250). Letters of invitation were mailed to 3,000 individuals. In total, 405 individuals (14.4%) contacted the website and 277 (9.8%) provided complete responses. The prepaid cash incentive generated the highest contact and response rates (23.3 and 17.3%, respectively), and no incentive generated the lowest (9.1 and 5.7%, respectively). The high lottery, however, was the most cost-effective incentive for obtaining completed surveys: compared with no incentive, the incremental cost-effectiveness ratio (ICER) per completed survey for high lottery was $13.89; for prepaid cash, the ICER was $18.29. This finding suggests that the preferred response incentive for community-based, stated-preference surveys is a lottery with a small number of large prizes.     

Primer postcard improves postal survey response rates

OBJECTIVE: This study measures the effect of an intervention to improve mailed survey response rates. METHOD: A randomised controlled trial of a 'primer' postcard was performed as part of a large survey in Victoria in 1997. Prior to the survey mailout, half the sample of 800 general practitioners supplied by the Health Insurance Commission was sent, at random, a primer card to request prompt return of the survey. RESULTS: The intervention resulted in a more rapid return of the survey and improved overall response rates from 60% to 66%. The increased cost per returned survey (40 cents) was largely offset by fewer non-responders requiring follow-up. CONCLUSIONS: A primer postcard is a time and cost-efficient method to increase response rates in general practitioner surveys. IMPLICATIONS: Public health researchers should consider implementing this intervention to improve response rates to postal surveys. Reports of other response maximising strategies should report the cost per returned survey to allow better comparison.     

A randomized trial of the impact of certified mail on response rate to a physician survey, and a cost-effectiveness analysis

This study's goals were to (a) determine whether sending a survey by certified mail results in a higher response rate from physicians compared to sending by first-class mail and (b) evaluate the cost-effectiveness of this method. The study sample was 409 physicians who were nonrespondents to two previous mailings of a medical specialty society survey. Eligible physicians were designated at random to receive a final mailing either by U.S. Postal Service certified mail including a return-receipt postcard or by first-class mail. There was a higher response rate from the certified mail group compared with the first-class mail group (41.3% versus 24.8%; relative risk = 1.66, 95% Confidence interval 1.25, 2.21). A cost-effectiveness analysis showed that the cost per respondent was higher using certified mail versus first-class mail in the third mailing ($2.77 versus $2.34). Thus, use of certified mail is effective in increasing survey response but more costly.     

PMD1 Information Technology in a Large Multi-Site Outcomes Study

The platinum chemotherapeutic compounds (cisplatin and carboplatin) are widely used in the treatment of advanced Non-Small Cell Lung Cancer (NSCLC). The introduction of new agents such as paclitaxel and vinorelbine has resulted in the development of combination regimens with improved response rates and survival. Two commonly used regimens, paclitaxel/carboplatin (TP) and vinorelbine/cisplatin (NP), are compared in this pharma-coeconomic analysis.
METHODS: A meta-analysis of available clinical trials was conducted to estimate the clinical effectiveness of TP and NP. Literature and physician interviews provided information on resource utilization and adverse event management (AEM) for these regimens. Treatment models were populated with Medicare reimbursement figures to compare the expected cost of treatment.
RESULTS: The expected cost of the TP and NP regimens was $19,322 and $20,790, respectively. Although the efficacy of these regimens has not been compared in a randomized trial, meta-analysis of regimented phase II and III studies showed no statistically significant differences in response rates. Therefore, equivalent efficacy is assumed in this cost comparison. A 20% variation in the cost of underlying resources yielded a 7% standard deviation in results. This sensitivity analysis showed that the costs of these regimens are insensitive to variations in underlying parameters.
CONCLUSION: This study suggests that TP is the phar-macoeconomic NSCLC treatment of choice when compared to NP. The analysis reveals that low administration and AEM costs are the key drivers in the lower treatment cost of TP.     

A monetary incentive increases postal survey response rates for pharmacists

BACKGROUND: Achieving acceptable response rates from health care providers via postal questionnaires is an ongoing challenge. The use of monetary incentives is one of the most effective strategies for increasing response rates. However, the effect and cost of such an incentive on retail pharmacists' response rates has not been well studied. METHODS: A sample of 700 pharmacies was selected at random from the electronic Yellow Pages in NSW Australia and mailed a brief survey regarding pharmacotherapies and advice for smoking cessation. Half of the sample was randomly allocated to receive an offer of an 14 US dollars gift voucher. RESULTS: The response rates were 65.9% for the voucher group and 53.5% for the no-voucher group. The odds of response from the voucher group was 1.68 (95%CI = 1.23, 2.30) times greater than for the no-voucher group. The cost per additional respondent was 67.95 US dollars. The incentive also reduced follow up costs by 10%. CONCLUSIONS: A moderately sized monetary incentive is able to achieve a significant increase in response rates for retail pharmacists, thereby reducing potential bias in the sample.     

Cost-effectiveness of inpatient substance abuse treatment.

OBJECTIVE: To identify the characteristics of cost-effective inpatient substance abuse treatment programs. DATA SOURCES/STUDY SETTING: A survey of program directors and cost and discharge data for study of 38,863 patients treated in 98 Veterans Affairs treatment programs. STUDY DESIGN: We used random-effects regression to find the effect of program and patient characteristics on cost and readmission rates. A treatment was defined as successful if the patient was not readmitted for psychiatric or substance abuse care within six months. PRINCIPAL FINDINGS: Treatment was more expensive when the program was smaller, or had a longer intended length of stay (LOS) or a higher ratio of staff to patients. Readmission was less likely when the program was smaller or had longer intended LOS; the staff to patient ratio had no significant effect. The average treatment cost $3,754 with a 75.0% chance of being effective, a cost-effectiveness ratio of $5,007 per treatment success. A 28-day treatment program was $860 more costly and 3.3% more effective than a 21-day program, an incremental cost-effectiveness of $26,450 per treatment success. Patient characteristics did not affect readmission rates in the same way they affected costs. Patients with a history of prior treatment were more likely to be readmitted but their subsequent stays were less costly. CONCLUSIONS: A 21-day limit on intended LOS would increase the cost-effectiveness of treatment programs. Consolidation of small programs would reduce cost, but would also reduce access to treatment. Reduction of the staff to patient ratio would increase the cost-effectiveness of the most intensively staffed programs.