Non-response bias in a sample survey on alcohol consumption

In a non-response follow-up study, non-respondents of the original mailed questionnaire were approached again by house visits in order to compare their alcohol consumption with that of the respondents of the same mailed questionnaire. Differences in alcohol consumption between respondents and non-respondents were found. There is strong evidence for overrepresentation of non-response among abstainers, but weak evidence among frequent excessive drinkers.     

Non-response and related factors in a nation-wide health survey

Objective: To analyse selective factors associated with an unexpectedly low response rate. Subjects and methods: The baseline questionnaire survey of a large prospective follow-up study on the psychosocial health of the Finnish working-aged randomly chosen population resulted in 21,101 responses (40.0%) in 1998. The non-respondent analysis used demographic and health-related population characteristics from the official statistics and behavioural, physical and mental health-related outcome differences between early and late respondents to predict possible non-response bias. Reasons for non-response, indicated by missing responses of late respondents, and factors affecting the giving of consent were also analysed. Results: The probability of not responding was greater for men, older age groups, those with less education, divorced and widowed respondents, and respondents on disability pension. The physical health-related differences between the respondents and the general population were small and could be explained by differences in definitions. The late respondents smoked and used more psychopharmaceutical drugs than the early ones, suggesting similar features in non-respondents. The sensitive issues had a small effect on the response rate. The consent to use a medical register-based follow-up was obtained from 94.5% of the early and 90.9% of the late respondents (odds ratio: 1.70; 95% confidence interval: 1.49–1.93). Consent was more likely among respondents reporting current smoking, heavy alcohol use, panic disorder or use of tranquillisers. Conclusions: The main reasons for non-response may be the predisposing sociodemographic and behavioural factors, the length and sensitive nature of the questionnaire to some extent, and a suspicion of written consent and a connection being made between the individual and the registers mentioned on the consent form.     

Characteristics of respondents and non-respondents from a case-control study of breast cancer in younger women

BACKGROUND: This study assessed the nature of potential biases by comparing respondents with non-respondents from a case-control study of breast cancer in younger women. METHODS: The case-control study was conducted in three regions in the US: Atlanta GA, Seattle/Puget Sound WA, and central New Jersey. An abbreviated interview or mailed questionnaire was completed by willing non-respondents, most of whom had refused participation in the main study. RESULTS: Respondents and non-respondents appeared similar with respect to age, race, relative weight, smoking, family history of breast cancer, number of births, age at first birth, and several dietary items. Compared to non-respondents, case and control respondents were of shorter stature, and reported less frequent consumption of doughnuts/pastries. Respondent cases, compared with non-respondent cases, were more highly educated and more likely to have consumed alcohol regularly; similar but not statistically significant tendencies were observed for controls. Respondent cases experienced menarche earlier than non-respondents. Respondent controls were more likely to have used oral contraceptives than non-respondents; a similar but not statistically significant tendency was observed in cases. Comparisons of crude and simulated relative risks using available non-respondents' data generally showed a low impact of non-response on relative risks in this study. CONCLUSIONS: Our results suggest that non-response would not greatly affect relative risk estimates in this study, except possibly regarding height. However, we were limited by the numbers of informative non-respondents and the amount of data collected. Collecting similar information in future studies would be useful, especially since varying methods used to encourage participation may lead to differences in respondents' characteristics.     

Characteristics of non-response in the Danish Health Interview Surveys, 1987-1994

BACKGROUND: The types and quantity of non-response in surveys influence the extent to which the results may be generalized. This study analysed trends in non-response in the Danish Health Interview Surveys from 1987 to 1994 and used the National Patient Registry to assess whether non-response biased the estimated population prevalence of morbidity when solely based on responders. METHODS: The data were for the 23,096 adults sampled for the Danish Health Interview Surveys in 1987, 1991 and 1994. All were followed using the National Patient Registry to obtain such information as hospital admissions. RESULTS: Non-response increased from 20.0% in 1987 to 22.6% in 1994. Four combinations of background variables characterized the non-response: gender and age; gender and civil status; county of residence and age; survey year and age. Non-respondents and respondents had identical gender- and age-standardized hospital admission rates for approximately 5 years before and 2 years after data collection, but non-respondents had a significantly higher rate immediately before and during data collection. Admissions rates were analysed according to reasons for non-response. Refusers had a lower admission rate than respondents before data collection but similar during and after data collection. The rate was higher during the whole period among ill or disabled non-respondents. Among people who could not be contacted during the data collection period a higher admission rate was only found immediately before and during data collection. CONCLUSIONS: Although admission rates differed between respondents and non-respondents these differences were too small to bias the estimated population prevalence of morbidity when solely based on respondents.     

Recruitment methods and differences in early, late and non-respondents in the first MONICA-Catalonia population survey

The early response rate in the first MONICA-Catalonia population survey was 52.7% and the final response rate was 73.8%. The intensity of recruitment effort in this survey led to a considerable increase in response rate (20%), with the extra cost per late respondent being relatively low ($13.9). Added recruitment effort was most effective in the youngest age group, 25-34 years. It was also more effective among women living in urban areas than among those from rural areas. In men, early respondents had a higher proportion of smokers than late respondents, and in women, early respondents had higher systolic and diastolic blood pressure levels and were more aware of their history of high blood pressure than late respondents. Non-respondents were less educated than respondents in both sexes, and this was more marked in women. No differences were found in the proportion of smokers between respondents and non-respondents. Respondents were more aware of their high blood pressure history than non-respondents. The recruitment costs and distribution of non-response components are given.     

Medical consumption of respondents and non-respondents to a mailed health survey

Non-response bias can distort the results of health surveys.The occurrence of selective non-response can be assessed whendata are available for both respondents and non-respondents.The objective of this study was to compare the medical consumptionof respondents and non-respondents to a mailed health survey.A mailed health survey was conducted among approximately 13,500adults and among parents of approximately 1,500 children aged5–15 years. The net response rate was 70.4%. A panel dataset that could be matched with the health survey data was availablefor all eligible persons. This data set comprises administrativeinformation on hospitalizations, annual health care expendituresand demographic variables. The results of this study show thatresponse was associated with age, sex, degree of urbanizationand type of insurance. After correcting for differences in demographicvariables, respondents and non-respondents differ in the utilizationof several types of care. Relatively more users than non-usersresponded. Response was not associated with the utilizationof care related to severe conditions such as in-patient hospitalcare. The conclusion from this study is that when a mailed healthsurvey is used to measure medical consumption, the non-responsebias will result in a small overestimation of utilization.     

Generalizability of valuations on health states collected with the EuroQolc-questionnaire

Problem: Non-response and non-usable response were found in population surveys on valuation of health states. If non-response is selective regarding valuations, then generalization of the resulting values to the whole survey population is not permitted. This could limit the use of empirical utility values in resource allocation in health care. Methods: Response behaviour of a sample of 1400 from the Dutch general population to the mailed EuroQol^c-questionnaire was analyzed by four methods. I. Phoning resolute non-respondents; II. comparison of zip code characteristics of respondents and non-respondents (because individual data on background characteristics were not available for the non-respondents); III. analysis of response over time (wave-analysis); IV: comparison of background variables of successful (less than two valuations missing) and unsuccessful respondents, combined with analysis of the effect of these background variables on valuations. Results: No indications for selective non-response were found, although the phenomenon appeared hard to investigate. The successful response came from a slightly younger and better educated subsample. However, a general influence of age and educational level on valuations could not be shown. This finding is consistent with the literature. Conclusion: Although the existence of selective non-response cannot be excluded, its relevance can be considered to be small. This finding is encouraging for the use of empirical utility values in allocative decisions.     

Postal survey on airborne occupational exposure and respiratory disorders in Norway: causes and consequences of non-response.

STUDY OBJECTIVE--The aim was to examine causes for non-response in a community survey, and how non-response influences prevalence estimates of some exposure and disease variables, and associations between the variables. DESIGN--This was a cross sectional questionnaire study with two reminder letters. The questionnaire asked for information on smoking habits, occupational airborne exposure and respiratory disorders. SETTING--A random sample of 4992 subjects from the general population aged 15-70 years of Hordaland County, Norway. MAIN RESULTS--The overall response rate was 90%, with a 63% response to the initial letter. The response rates to the first and second reminder letters were 56% and 36% respectively. In 20% of the non-respondents an uncompleted questionnaire was returned with cause for non-response; in two thirds of these the cause for non-response was that the subject was not resident at the mailing address. A home visit to a random sample of 50 urban non-respondents provided further information on 29 subjects. A wrong address at the Central Population Registry and the subject's feeling of lack of personal benefit from a postal survey were the major reasons for non-response. Smokers were late respondents and subjects with respiratory disorders tended to be early respondents. CONCLUSION--The main reasons for non-response were a wrong mailing address and a feeling of lack of personal benefit from responding. Using only the initial letter would have changed the estimated prevalence of smokers from 39% to 35%. Otherwise, the estimated prevalence of the exposure and disease variables as well as the associations between them were only slightly changed after including the respondents to the first and second reminder letters.     

Effect on Trend Estimates of the Difference between Survey Respondents and Non-respondents: Results from 27 Populations in the WHO MONICA Project

Introduction: In the World Health Organization (WHO) MONICA (multinational MONItoring of trends and determinants in CArdiovascular disease) Project considerable effort was made to obtain basic data on non-respondents to community based surveys of cardiovascular risk factors. The first purpose of this paper is to examine differences in socio-economic and health profiles among respondents and non-respondents. The second purpose is to investigate the effect of non-response on estimates of trends. Methods:Socio-economic and health profile between respondents and non-respondents in the WHO MONICA Project final survey were compared. The potential effect of non-response on the trend estimates between the initial survey and final survey approximately ten years later was investigated using both MONICA data and hypothetical data. Results: In most of the populations, non-respondents were more likely to be single, less well educated, and had poorer lifestyles and health profiles than respondents. As an example of the consequences, temporal trends in prevalence of daily smokers are shown to be overestimated in most populations if they were based only on data from respondents. Conclusions: The socio-economic and health profiles of respondents and non-respondents differed fairly consistently across 27 populations. Hence, the estimators of population trends based on respondent data are likely to be biased. Declining response rates therefore pose a threat to the accuracy of estimates of risk factor trends in many countries.     

Predicting risk among non-respondents in prospective studies

Potential non-response bias was investigated in a follow-up study of 2,011 chronically disabled patients. 82.5% and 73.3% of the study subjects responded to self-administered mail questionnaires respectively at 6-month and 1-year follow-up. Information on employment status, the outcome of interest, of approximately 90% of the non-respondents was obtained from indirect sources. Employment rate was lower among the non-respondents than the respondents. Non-response was associated with age, social class, previous employment record, and the type of disability; but none of these characteristics were associated with the outcome. Out of the five known independent risk factors for unemployment, only one (incompletion of rehabilitation course) was associated with non-response. The employment rate among the respondents was also assessed according to the delay in response, that is the number of reminders sent to achieve response. The outcome among- the late respondents was similar to that among the nonrespondents. These data suggest that (a) risk estimates may be biased even when the response rate is greater than 80%, (b) the prevalence of risk factors among non-respondents may not indicate the presence or the degree of non-response bias, but (c) reliable estimates can be obtained from extrapolations of the rates among the respondents according to the delay in response.     

Survey non-response in the Netherlands: effects on prevalence estimates and associations

PURPOSE: Differences in respondent characteristics may lead to bias in prevalence estimates and bias in associations. Both forms of non-response bias are investigated in a study on psychosocial factors and cancer risk, which is a sub-study of a large-scale monitoring survey in the Netherlands. METHODS: Respondents of a cross-sectional monitoring project (MORGEN; N = 22,769) were also asked to participate in a prospective study on psychosocial factors and cancer risk (HLEQ; N = 12,097). To investigate diverse aspects of non-response in the HLEQ on prevalence estimates and associations are studied, based on information gathered in the MORGEN-project. RESULTS: A response percentage of 45% was obtained in the MORGEN-project. Response rates were found to be lower among men and younger people. The HLEQ showed a response percentage of 56%, and respondents reported higher socioeconomic status, better subjective health and healthier lifestyle behaviors than non-respondents. However, associations between smoking status and either socioeconomic status or subjective health based on respondents only were not statistically different from those based on the entire MORGEN-population. CONCLUSION: Non-response leads to bias in prevalence estimates of current smoking, current alcohol intake, and low physical activity or poor subjective health. However, non-response did not cause bias in the examined associations.     

Response patterns to a postal survey using a cervical screening register as the sampling frame

This study compared different types of respondent to a postal survey. A random sample of women aged 20-64 years (n = 4057) was selected from a population-based cervical screening register to examine their information experience during the screening programme. The initial response rate was 57%, and this increased to 81% after one reminder. Respondents were older (P < 0.0001) than non-respondents, but both groups were comparable with regard to attendance history for cervical screening (respondents 89.3%; non-respondents 89.1%) and normal smear test results (respondents 84.0%; non-respondents 81.4%). Early and late respondents were similar in age. Compared with late respondents, early respondents were more likely to live in highly affluent (P < 0.0001) and rural areas (P = 0.026). They were also more likely to be non-attenders (P < 0.0001), but they were less likely to have had inadequate smear results (P = 0.030) than late respondents. These results suggest that consideration should be given to factors other than sociodemographic differences when examining response patterns to postal surveys.     

The effect of non-response on estimates of health care utilisation: linking health surveys and registers

BACKGROUND: Non-response in health surveys may lead to bias in estimates of health care utilisation. The magnitude, direction and composition of the bias are usually not well known. When data from health surveys are merged with data from registers at the individual level, analyses can reveal non-response bias. Our aim was to estimate the composition, direction and magnitude of non-response bias in the estimation of health care costs in two types of health interview surveys. METHODS: The surveys were (1) a national personal interview survey of 22 484 Danes (2) a telephone interview survey of 5000 Danes living in Funen County. Data were linked with register information on health care utilisation in hospitals and primary care. Health care utilisation was estimated for respondents and non-respondents, and the difference was explained by a decomposition method of bias components. RESULTS: The surveys produced the same pattern of non-response, but with slight differences in non-response bias. Response rates for the interview and telephone surveys were 75 and 69%, respectively. Refusal was the most frequent reason for non-response (22 and 20% of those sampled, respectively), whereas illness, non-contact, and other reasons were less frequent. Respondents used 3-6% less health care than non-respondents at the aggregate level, but the opposite was true for some specific types of health care. Non-response due to illness was the main contributor to non-response bias. CONCLUSIONS: Different types of non-response have different bias effects. However, the magnitude of the bias encourages the continued use of interview health surveys.     

Estimating Non-Response Bias in Family Studies: Application to Mental Health and Lifestyle

Non-response to mailed surveys reduces the effective sample size and may introduce bias. Non-response has been studied by (1) comparison to available data in population based registers, (2) directly contacting non-respondents by telephone or single-item reply cards, and (3) longitudinal repetition of the survey. The goal of this paper was to propose an additional method to study non-response bias: when the variable of interest has a familial component, data from respondents can be used as proxy for the data from their non-responding family members. This approach was used with data on smoking, alcohol consumption, physical activity, coffee- and tea-use, education, body mass index, religion, burnout, life events, personality and mental health in large number of siblings and DZ twins registered with the Netherlands Twin Register. In addition, for smoking behavior, we also used the second strategy by sending a reply card. Results show that scores of members from less cooperative families or incomplete twin pairs tended to be more unfavorable than the scores from highly cooperative families or complete twin pairs. For example, family members from less cooperative families cycled less often and scored higher on anxious depression and neuroticism. For smoking, both the results of the reply card and the results of the additional method suggested a higher percentage smokers among the non-respondents but this was only significant with reply card method. In general, differences between highly/less cooperative families and complete/incomplete DZ twins were small. Results suggest that, even for studies with moderate response rates, data collected on health, personality and lifestyle are relatively unbiased.     

On weighting the rates in non-response weights

A basic estimation strategy in sample surveys is to weight units inversely proportional to the probability of selection and response. Response weights in this method are usually estimated by the inverse of the sample-weighted response rate in an adjustment cell, that is, the ratio of the sum of the sampling weights of respondents in a cell to the sum of the sampling weights for respondents and non-respondents in that cell. We show by simulations that weighting the response rates by the sampling weights to adjust for design variables is either incorrect or unnecessary. It is incorrect, in the sense of yielding biased estimates of population quantities, if the design variables are related to survey non-response; it is unnecessary if the design variables are unrelated to survey non-response. The correct approach is to model non-response as a function of the adjustment cell and design variables, and to estimate the response weight as the inverse of the estimated response probability from this model. This approach can be implemented by creating adjustment cells that include design variables in the cross-classification, if the number of cells created in this way is not too large. Otherwise, response propensity weighting can be applied.     

Non-participation and mortality in different socioeconomic groups: the FINRISK population surveys in 1972-92

BACKGROUND: Declining response rates pose a serious threat to the validity of estimates derived from epidemiological studies. If respondents and non-respondents differ systematically from each other, there can be a bias in the results of the study. A population-based cohort study was conducted to investigate disparities in socioeconomic structure between respondents and non-respondents and the contribution of these disparities to socioeconomic differences in total and cardiovascular mortality. DESIGN: Data comprised 32,354 male and female participants and 4890 non-participants aged 35-74 years who belonged to the sample in one of the five FINRISK surveys in 1972, 1977, 1982, 1987 or 1992 in Finland. They were followed up for 9 years and 6 months. RESULTS: It was found that the lower socioeconomic groups were over-represented among non-respondents both in men and women. When comparing the relative risk of death using the highest socioeconomic group of the participants as the reference group, it was found that although the socioeconomic gradient was similar for participants and non-participants-that is, lower groups had a higher risk of death-the risk was at a higher level among non-respondents. CONCLUSIONS: Basing analysis on participants does not distort the relative risk of death associated with socioeconomic position. However, it does underestimate the absolute risk.     

Does survey non-response bias the association between occupational social class and health?

AIMS: A non-response rate of 20-40%is typical in questionnaire studies. The authors evaluate non-response bias and its impact on analyses of social class inequalities in health. METHODS: Set in the context of a health survey carried out among the employees of the City of Helsinki (non-response 33%) in 2000-02. Survey response and non-response records were linked with a personnel register to provide information on occupational social class and long sickness absence spells as an indicator of health status. RESULTS: Women and employees in higher occupational social classes were more likely to respond. Non-respondents had about 20-30% higher sickness absence rates. Relative social class differences in sickness absence in the total population were similar to those among either respondents or non-respondents. CONCLUSIONS: In working populations survey non-response does not seriously bias analyses of social class inequalities in sickness absence and possibly health inequalities more generally.     

Characteristics of respondents and non-respondents to a mailed questionnaire.

In establishing a cohort of U.S. nurses, an assessment of response bias was made comparing respondents and non-respondents with regard to age, education, state of residence, employment status, field of employment, and major specialty. Overall, the 122,328 respondents (69.7 per cent) and 43,222 non-respondents were quite similar. Together with the reasonable response rate in a homogeneous population, this suggests that estimation of exposure-disease associations is unlikely to be affected by major bias due to non-response.     

Marital status,educational level and household income explain part of the excess mortality of survey non-respondents

Survey respondents and non-respondents differ in their demographic and socio-economic position. Many of the health behaviours are also known to be associated with socio-economic differences. We aimed to investigate how much of the excess mortality of survey non-respondents can be explained by the socio-economic differences between respondents and non-respondents. Questionnaire-based adult health behaviour surveys have been conducted in Finland annually since 1978. Data from the 1978 to 2002 surveys, including non-respondents, were linked with mortality data from the Finnish National Cause of Death statistics and with demographic and socio-economic register data (marital status, education and household income) obtained from Statistics Finland. The mortality follow-up lasted until 2006, in which period there were 12,762 deaths (7,994 in men and 4,768 in women) during the follow-up. Total and cause-specific mortality were higher among non-respondents in both men and women. Adjusting results for marital status, educational level and average household income decreased the excess total and cause-specific mortality of non-respondents in both men and women. Of the total excess mortality of non-respondents, 41% in men and 20% in women can be accounted for demographic and socio-economic factors. A part of the excess mortality among non-respondents can be accounted for their demographic and socio-economic characteristics. Based on these results we can assume that non-respondents tend to have more severe health problems, acute illnesses and unhealthy behaviours, such as smoking and excess alcohol use. These can be reasons for persons not taking part in population surveys.     

Non-response bias in a web-based health behaviour survey of New Zealand tertiary students

ObjectiveThere has been little investigation of non-response bias in web-based health surveys. We hypothesised that non-respondents have a higher prevalence of risk behaviours than respondents.MethodIn 2005, random samples of students aged 17–25 years from 12 New Zealand tertiary institutions (n = 7130) were invited to complete a web-based health behaviour survey, with three e-mail reminders. Early respondents (before 2nd reminder) were compared with late respondents (after 2nd reminder). Late respondents served as a proxy for non-respondents.Results2607 students (37%) responded early, 676 (9%) responded late, and 3847 (54%) did not respond. There were differences between early and late respondents in high school binge drinking (38% vs 47%, p = 0.002) and non-compliance with physical activity guidelines (12% vs 18%, p = 0.004). Differences in overweight/obesity (26% vs 31%, p = 0.058), smoking (18% vs 22%, p = 0.091) and non-compliance with dietary guidelines (76% vs 77%, p = 0.651) were non-significant but point estimates were in the expected direction. Estimated bias in prevalence of risk behaviours was an absolute difference of 1–4% and a relative difference of 0–21%.ConclusionRespondents whose participation was hardest to elicit reported more risk behaviour. Assuming non-respondents' behaviour is similar or more extreme than that of late respondents, prevalence will have been substantially underestimated.