New outcome rates for random sampling telephone surveys

BackgroundSurvey non-response rates are important quality indicators. Refusal rates can induce non-response bias in health survey estimates. However, comparisons across surveys highlight inconsistencies in the use of survey outcome categories and in the calculation of response rates. In this paper we discuss the relevance of these indicators and suggest other survey quality indicators.MethodsOutcome rates from two French random-digit dialing (RDD) telephone surveys are compared : the Nicolle survey on infectious diseases of 4112 individuals conducted in 2006, and the HIV knowledge, attitude, belief and practices (KABP) survey of 5071 individuals in 2004. Based on the same protocol, we describe in details the way the two RDD samples were drawn and how non-response rates were estimated.ResultsNon-response rates were different: 36% in Nicolle survey and 18% in KABP survey. However, the quantity of telephone numbers required to obtain one interview was higher in the KABP survey: 2.8 telephone numbers versus 2.1 in the Nicolle survey. The participation rates, aggregating together refusals, break-off and non-reachable numbers, were equivalent for the two surveys. This result occurred because of a greater proportion of unreached calls in the KABP surveys, which is not integrated into the non-response rates commonly used.ConclusionSurvey non-response rate is insufficient to estimate the quality of a survey. The need for other indicators has been previously stressed in the literature, notably with the adoption and utilization of the American Association for Public Opinion Research (AAPOR) standard definitions of four indicators. But these indicators are quite complex for evaluating non-response bias between surveys. In addition to the classical refusal rate, two other indicators are proposed in this paper: participation rate (number of complete interviews divided by the number of eligible and of unknown eligibility units) and a liking contact rate (number of unreachable units because of a long absence, break-off or non-answer divided by the number of eligible and of unknown eligibility units). The sum of these three indicators is equal to 100% and thus easier to manipulate when comparing surveys.