Estimating non-response bias in a survey on alcohol consumption: comparison of response waves

AIMS: According to 'the continuum of resistance model' late respondents can be used as a proxy for non-respondents in estimating non-response bias. In the present study, the validity of this model was explored and tested in three surveys on alcohol consumption. METHODS: The three studies collected their data by means of mailed questionnaires on alcohol consumption whereby two studies also performed a non-response follow-up. RESULTS: Comparisons of early respondents, late respondents and non-respondents in one study showed some support for 'the continuum of resistance model', although another study could not confirm this result. Comparison of alcohol consumption between three time response groups showed no significant linear pattern of differences between response waves. CONCLUSIONS: The hypothesis that late respondents are more similar to non-respondents than early respondents, could not be confirmed or rejected. Repeated mailings are effective in obtaining a greater sample size, but seem ineffective in improving the representativeness of alcohol consumption surveys.     

Anonymous non-response analysis in the ABCD cohort study enabled by probabilistic record linkage

Selective non-response is an important threat to study validity as it can lead to selection bias. The Amsterdam Born Children and their Development study (ABCD-study) is a large cohort study addressing the relationship between life style, psychological conditions, nutrition and sociodemographic background of pregnant women and their children's health. Possible selective non-response and selection bias in the ABCD-study were analysed using national perinatal registry data. ABCD-study data were linked with national perinatal registry data by probabilistic medical record linkage techniques. Differences in the prevalence of relevant risk factors (sociodemographic and care-related factors) and birth outcomes between respondents and non-respondents were tested using Pearson chi-squared tests. Selection bias (i.e. bias in the association between risk factors and specific outcomes) was analysed by regression analysis with and without adjustment for participation status.
The ABCD non-respondents were significantly younger, more often non-western, and more often multiparae. Non-respondents entered antenatal care later, were more often under supervision of an obstetrician and had a spontaneous delivery more often. Non-response however, was not significantly associated with preterm birth (odds ratio 1.10; 95% CI 0.93, 1.29) or low birthweight (odds ratio 1.16; 95% CI 0.98, 1.37) after adjustment for sociodemographic risk factors. The associations found between risk factors and adverse pregnancy outcomes were similar for respondents and non-respondents. Anonymised record linkage of cohort study data with national registry data indicated that selective non-response was present in the ABCD-study, but selection bias was acceptably low and did not influence the main study questions.     

Effect on Trend Estimates of the Difference between Survey Respondents and Non-respondents: Results from 27 Populations in the WHO MONICA Project

Introduction: In the World Health Organization (WHO) MONICA (multinational MONItoring of trends and determinants in CArdiovascular disease) Project considerable effort was made to obtain basic data on non-respondents to community based surveys of cardiovascular risk factors. The first purpose of this paper is to examine differences in socio-economic and health profiles among respondents and non-respondents. The second purpose is to investigate the effect of non-response on estimates of trends. Methods:Socio-economic and health profile between respondents and non-respondents in the WHO MONICA Project final survey were compared. The potential effect of non-response on the trend estimates between the initial survey and final survey approximately ten years later was investigated using both MONICA data and hypothetical data. Results: In most of the populations, non-respondents were more likely to be single, less well educated, and had poorer lifestyles and health profiles than respondents. As an example of the consequences, temporal trends in prevalence of daily smokers are shown to be overestimated in most populations if they were based only on data from respondents. Conclusions: The socio-economic and health profiles of respondents and non-respondents differed fairly consistently across 27 populations. Hence, the estimators of population trends based on respondent data are likely to be biased. Declining response rates therefore pose a threat to the accuracy of estimates of risk factor trends in many countries.     

Characteristics of non-response in the Danish Health Interview Surveys, 1987-1994

BACKGROUND: The types and quantity of non-response in surveys influence the extent to which the results may be generalized. This study analysed trends in non-response in the Danish Health Interview Surveys from 1987 to 1994 and used the National Patient Registry to assess whether non-response biased the estimated population prevalence of morbidity when solely based on responders. METHODS: The data were for the 23,096 adults sampled for the Danish Health Interview Surveys in 1987, 1991 and 1994. All were followed using the National Patient Registry to obtain such information as hospital admissions. RESULTS: Non-response increased from 20.0% in 1987 to 22.6% in 1994. Four combinations of background variables characterized the non-response: gender and age; gender and civil status; county of residence and age; survey year and age. Non-respondents and respondents had identical gender- and age-standardized hospital admission rates for approximately 5 years before and 2 years after data collection, but non-respondents had a significantly higher rate immediately before and during data collection. Admissions rates were analysed according to reasons for non-response. Refusers had a lower admission rate than respondents before data collection but similar during and after data collection. The rate was higher during the whole period among ill or disabled non-respondents. Among people who could not be contacted during the data collection period a higher admission rate was only found immediately before and during data collection. CONCLUSIONS: Although admission rates differed between respondents and non-respondents these differences were too small to bias the estimated population prevalence of morbidity when solely based on respondents.     

Does survey non-response bias the association between occupational social class and health?

AIMS: A non-response rate of 20-40%is typical in questionnaire studies. The authors evaluate non-response bias and its impact on analyses of social class inequalities in health. METHODS: Set in the context of a health survey carried out among the employees of the City of Helsinki (non-response 33%) in 2000-02. Survey response and non-response records were linked with a personnel register to provide information on occupational social class and long sickness absence spells as an indicator of health status. RESULTS: Women and employees in higher occupational social classes were more likely to respond. Non-respondents had about 20-30% higher sickness absence rates. Relative social class differences in sickness absence in the total population were similar to those among either respondents or non-respondents. CONCLUSIONS: In working populations survey non-response does not seriously bias analyses of social class inequalities in sickness absence and possibly health inequalities more generally.     

Non-response bias in a sample survey on alcohol consumption

In a non-response follow-up study, non-respondents of the original mailed questionnaire were approached again by house visits in order to compare their alcohol consumption with that of the respondents of the same mailed questionnaire. Differences in alcohol consumption between respondents and non-respondents were found. There is strong evidence for overrepresentation of non-response among abstainers, but weak evidence among frequent excessive drinkers.     

Predicting risk among non-respondents in prospective studies

Potential non-response bias was investigated in a follow-up study of 2,011 chronically disabled patients. 82.5% and 73.3% of the study subjects responded to self-administered mail questionnaires respectively at 6-month and 1-year follow-up. Information on employment status, the outcome of interest, of approximately 90% of the non-respondents was obtained from indirect sources. Employment rate was lower among the non-respondents than the respondents. Non-response was associated with age, social class, previous employment record, and the type of disability; but none of these characteristics were associated with the outcome. Out of the five known independent risk factors for unemployment, only one (incompletion of rehabilitation course) was associated with non-response. The employment rate among the respondents was also assessed according to the delay in response, that is the number of reminders sent to achieve response. The outcome among- the late respondents was similar to that among the nonrespondents. These data suggest that (a) risk estimates may be biased even when the response rate is greater than 80%, (b) the prevalence of risk factors among non-respondents may not indicate the presence or the degree of non-response bias, but (c) reliable estimates can be obtained from extrapolations of the rates among the respondents according to the delay in response.     

Non-response and related factors in a nation-wide health survey

Objective: To analyse selective factors associated with an unexpectedly low response rate. Subjects and methods: The baseline questionnaire survey of a large prospective follow-up study on the psychosocial health of the Finnish working-aged randomly chosen population resulted in 21,101 responses (40.0%) in 1998. The non-respondent analysis used demographic and health-related population characteristics from the official statistics and behavioural, physical and mental health-related outcome differences between early and late respondents to predict possible non-response bias. Reasons for non-response, indicated by missing responses of late respondents, and factors affecting the giving of consent were also analysed. Results: The probability of not responding was greater for men, older age groups, those with less education, divorced and widowed respondents, and respondents on disability pension. The physical health-related differences between the respondents and the general population were small and could be explained by differences in definitions. The late respondents smoked and used more psychopharmaceutical drugs than the early ones, suggesting similar features in non-respondents. The sensitive issues had a small effect on the response rate. The consent to use a medical register-based follow-up was obtained from 94.5% of the early and 90.9% of the late respondents (odds ratio: 1.70; 95% confidence interval: 1.49–1.93). Consent was more likely among respondents reporting current smoking, heavy alcohol use, panic disorder or use of tranquillisers. Conclusions: The main reasons for non-response may be the predisposing sociodemographic and behavioural factors, the length and sensitive nature of the questionnaire to some extent, and a suspicion of written consent and a connection being made between the individual and the registers mentioned on the consent form.     

Postal survey on airborne occupational exposure and respiratory disorders in Norway: causes and consequences of non-response.

STUDY OBJECTIVE--The aim was to examine causes for non-response in a community survey, and how non-response influences prevalence estimates of some exposure and disease variables, and associations between the variables. DESIGN--This was a cross sectional questionnaire study with two reminder letters. The questionnaire asked for information on smoking habits, occupational airborne exposure and respiratory disorders. SETTING--A random sample of 4992 subjects from the general population aged 15-70 years of Hordaland County, Norway. MAIN RESULTS--The overall response rate was 90%, with a 63% response to the initial letter. The response rates to the first and second reminder letters were 56% and 36% respectively. In 20% of the non-respondents an uncompleted questionnaire was returned with cause for non-response; in two thirds of these the cause for non-response was that the subject was not resident at the mailing address. A home visit to a random sample of 50 urban non-respondents provided further information on 29 subjects. A wrong address at the Central Population Registry and the subject's feeling of lack of personal benefit from a postal survey were the major reasons for non-response. Smokers were late respondents and subjects with respiratory disorders tended to be early respondents. CONCLUSION--The main reasons for non-response were a wrong mailing address and a feeling of lack of personal benefit from responding. Using only the initial letter would have changed the estimated prevalence of smokers from 39% to 35%. Otherwise, the estimated prevalence of the exposure and disease variables as well as the associations between them were only slightly changed after including the respondents to the first and second reminder letters.     

Characteristics of respondents and non-respondents to a mailed questionnaire.

In establishing a cohort of U.S. nurses, an assessment of response bias was made comparing respondents and non-respondents with regard to age, education, state of residence, employment status, field of employment, and major specialty. Overall, the 122,328 respondents (69.7 per cent) and 43,222 non-respondents were quite similar. Together with the reasonable response rate in a homogeneous population, this suggests that estimation of exposure-disease associations is unlikely to be affected by major bias due to non-response.     

Comparison of telephone sampling and area sampling: response rates and within-household coverage

Random digit dialing is used frequently in epidemiologic case-control studies to select population-based controls, even when both cases and controls are interviewed face-to-face. However, concerns persist about the potential biases of random digit dialing, particularly given its generally lower response rates. In an Atlanta, Georgia, case-control study of breast cancer among women aged 20-54 years, all of whom were interviewed face-to-face, two statistically independent control groups were compared: those obtained through random digit dialing (n = 652) and those obtained through area probability sampling (n = 640). The household screening rate was significantly higher for the area sample, by 5.5%. Interview response rates were comparable. The telephone sample estimated a significantly larger percentage (by approximately 7%) of households to have no age-eligible women. Both control groups, appropriately weighted, had characteristics similar to US Census demographic characteristics for Atlanta women, except that respondents in both control groups were more educated and more likely to be married. The authors conclude that households contacted through random digit dialing are somewhat less likely to participate in the household screening process, and if they are cooperative, some households may not disclose that age-eligible women reside therein. Investigators need to develop improved methods for screening and enumerating household members in random digit dialing surveys that target a specific subpopulation, such as women.     

Survey non-response in the Netherlands: effects on prevalence estimates and associations

PURPOSE: Differences in respondent characteristics may lead to bias in prevalence estimates and bias in associations. Both forms of non-response bias are investigated in a study on psychosocial factors and cancer risk, which is a sub-study of a large-scale monitoring survey in the Netherlands. METHODS: Respondents of a cross-sectional monitoring project (MORGEN; N = 22,769) were also asked to participate in a prospective study on psychosocial factors and cancer risk (HLEQ; N = 12,097). To investigate diverse aspects of non-response in the HLEQ on prevalence estimates and associations are studied, based on information gathered in the MORGEN-project. RESULTS: A response percentage of 45% was obtained in the MORGEN-project. Response rates were found to be lower among men and younger people. The HLEQ showed a response percentage of 56%, and respondents reported higher socioeconomic status, better subjective health and healthier lifestyle behaviors than non-respondents. However, associations between smoking status and either socioeconomic status or subjective health based on respondents only were not statistically different from those based on the entire MORGEN-population. CONCLUSION: Non-response leads to bias in prevalence estimates of current smoking, current alcohol intake, and low physical activity or poor subjective health. However, non-response did not cause bias in the examined associations.     

Potential for bias in case-control studies of oral contraceptives and breast cancer

The discrepancies between the findings of the 6 large case-control studies to study the association between oral contraceptive (OC) use and breast cancer diagnosed in the 1980s may be due to chance or bias. The likelihood that chance played a role is suggested by the large numbers of subgroups examined in each study, the inconsistencies in the findings of different studies, and the wide confidence intervals around most of the relative risks. The most serious potential problem in case-control studies is that the procedures used to select cases and controls may produce groups that are not truly comparable. Recall bias is likely to contaminate information about the duration and type of part OC use. In addition, the more frequent examination of the breasts of women using OCs can produce surveillance bias. 6 procedures are recommended to minimize bias in future case-control studies of OC use and breast cancer: 1) whenever possible, cases and controls should be selected from an entire community; 2) if hospital controls need to be used, there should be explicit criteria for selecting them and the proportions of OC users in each diagnostic group should be presented; 3) women interviewed should not be aware of the study's hypotheses; 4) interviewers should be kept blind as to whether a subject is a case or control; 5) the possibility of recall bias should be investigated by comparing contraceptive histories from a sample of cases and controls with an independent source of information (preferably recorded before cancers were diagnosed); and 7) the interview should include questions about the frequency of breast examinations, so that any effects of more frequent surveillance of OC users can be controlled for.     

Response patterns to a postal survey using a cervical screening register as the sampling frame

This study compared different types of respondent to a postal survey. A random sample of women aged 20-64 years (n = 4057) was selected from a population-based cervical screening register to examine their information experience during the screening programme. The initial response rate was 57%, and this increased to 81% after one reminder. Respondents were older (P < 0.0001) than non-respondents, but both groups were comparable with regard to attendance history for cervical screening (respondents 89.3%; non-respondents 89.1%) and normal smear test results (respondents 84.0%; non-respondents 81.4%). Early and late respondents were similar in age. Compared with late respondents, early respondents were more likely to live in highly affluent (P < 0.0001) and rural areas (P = 0.026). They were also more likely to be non-attenders (P < 0.0001), but they were less likely to have had inadequate smear results (P = 0.030) than late respondents. These results suggest that consideration should be given to factors other than sociodemographic differences when examining response patterns to postal surveys.     

An epidemiologic study of fibrocystic breast disease with reference to ductal epithelial atypia

A case-control study of 255 women with fibrocystic breast lesions and 790 controls was conducted at two hospitals in New Haven, Connecticut during 1977-1979. Cases were found to weigh significantly less than controls, and were more likely than controls to have: a first-degree female relative with a history of breast cancer; a higher level of education; a recent history of regular gynecologic checkups; and (if under age 45 years) a later age at first pregnancy. They were less likely to have had a surgical menopause. The degree of ductal epithelial atypia in breast biopsy specimens was evaluated in order to see whether epidemiologic characteristics differed according to the degree of ductal atypia. The only variable to show a linear relationship with ductal atypia was a recent history of regular gynecologic checkups; those with no or minimal atypia were more likely to have had recent checkups than those with high atypia scores. This study thus gives no evidence that known risk factors for breast cancer are more strongly associated with fibrocystic breast disease with a high degree of atypia than with fibrocystic breast disease with a low degree of atypia. It also provides data to support the belief that women having frequent gynecologic checkups are more likely to be included as cases in case-control studies of fibrocystic breast disease, and particularly in the groups with no or minimal atypia, than those not having frequent checkups.     

Reliability of data from proxy respondents in an international case-control study of cardiovascular disease and oral contraceptives. World Health Organization Collaborative Study of Cardiovascular Disease and Steroid Hormone Contraception.

STUDY OBJECTIVES: To evaluate the reliability of data supplied in a case-control study by proxy respondents for cases who were too ill to do so themselves. DESIGN: A hospital based, case-control study of the current use of oral contraceptives (OC) and cardiovascular diseases. Data from "true" controls matched to a subset of cases were compared with those supplied by proxy respondents about the true controls. SETTING: Hospitals in 21 centres from Africa, Asia, Europe, and Latin America. PATIENTS AND PARTICIPANTS: For a subset of cases, 403 pairs of controls-one "true" and one proxy-were interviewed. "True" controls were matched by age, place, and time of admission and were admitted with 1 of 27 permissible diagnoses not associated with OC use. Proxy controls were either relatives or friends of true controls. MAIN RESULTS: Levels of concordance between data from proxy and true controls were high for most variables regarding recent events, including current OC use, but were greatly diminished when detailed information, particularly from the past, was required. Husbands were usually the best proxy, although this was question-specific. The sensitivity and specificity of proxy responses were 93% (95% confidence intervals: 77%, 99%) and 100% (98%, 100%) respectively, for current use of OC. Assuming the misclassification of current OC use by proxy cases is similar to that produced by proxy controls, the estimated impact of using proxy data on risk estimates associated with current OC use was to bias the overall estimate of risk of stroke by less than 3% and the risks of both acute myocardial infarction and venous thromboembolism by less than 1%. CONCLUSIONS: Friends or relatives, and particularly husbands, provided reliable information when used as proxy respondents for young women. The estimated impact of misclassification by proxy respondents on overall risk estimates in the WHO collaborative study was less than that which would have arisen if information from proxy respondents had not been used.     

Anthropometric indicators of breast cancer risk in postmenopausal women in Hawaii

As part of a case-control study of breast cancer in Hawaii, self-reported data on height, weight at various ages, breast size, shoe size, and triceps skinfold thickness were collected from 138 Japanese postmenopausal cases, 134 Caucasian postmenopausal cases, 154 Japanese neighborhood controls, and 142 Caucasian neighborhood controls. In a multiple covariance analysis, cases of both ethnic groups were slightly heavier (at all ages) and more obese (based on a weight-corrected-for-height index) than were controls, although none of the differences was statistically significant. Among the Japanese only, cases were also taller, had a greater body surface area (computed from the height and weight data), and had a larger shoe size than did the controls. The latter finding was statistically significant (p less than 0.05). Odds ratios were computed by multiple logistic regression analysis and revealed no additional relationships; however, there were suggested dose-response gradients for height, weight at age 20, and body surface area in the Japanese women and for breast size in the Caucasian women. A further analysis of risks based on changes in relative body weight between young adult life (age 20) and current age was also negative. Overall, these findings offer only weak support for an association between weight or obesity and breast cancer risk and suggest that anthropometric indices are at best very indirect indicators of true etiologic factors for breast cancer.     

Do low control response rates always affect the findings? Assessments of smoking and obesity in two Australian case‐control studies of cancer

Objective: Participation rates have been declining in case-control studies, particularly among controls, raising concerns about possible bias. Formal assessments of the effect of low participation on odds ratios (OR) are seldom presented however. We sought to quantify possible bias using multiple imputation techniques.
Methods: Using data from two Australian case-control studies, we estimated the relative risks of oesophageal squamous cell carcinoma (OSCC) and adenocarcinoma (OAC), and serous ovarian cancer (SOC) associated with smoking and body mass index (BMI). We compared ORs observed using self-reported data from participating controls with ORs derived using imputed exposures for non-participating controls.
Results: Participating controls were less likely than non-participants to smoke currently. Smoking remained significantly associated with oesophageal cancer even under the most extreme assumption of smoking prevalence among non-participants (OSCC: observed OR 6.54, 4.62-9.28, imputed OR 3.94, 2.83-5.49; OAC: observed OR 2.69, 1.87-3.85 imputed OR 1.58, 1.13-2.22). For SOC however, risks associated with smoking were attenuated to null under plausible smoking assumptions among non-participants. BMI distributions were similar among participating and non-participating controls, and risk estimates were essentially unchanged.
Conclusion and implications: Bias is not an inevitable consequence of low control participation and depends on the association examined. Sensitivity analyses can assist in interpretation of results.     

Recruitment methods and differences in early, late and non-respondents in the first MONICA-Catalonia population survey

The early response rate in the first MONICA-Catalonia population survey was 52.7% and the final response rate was 73.8%. The intensity of recruitment effort in this survey led to a considerable increase in response rate (20%), with the extra cost per late respondent being relatively low ($13.9). Added recruitment effort was most effective in the youngest age group, 25-34 years. It was also more effective among women living in urban areas than among those from rural areas. In men, early respondents had a higher proportion of smokers than late respondents, and in women, early respondents had higher systolic and diastolic blood pressure levels and were more aware of their history of high blood pressure than late respondents. Non-respondents were less educated than respondents in both sexes, and this was more marked in women. No differences were found in the proportion of smokers between respondents and non-respondents. Respondents were more aware of their high blood pressure history than non-respondents. The recruitment costs and distribution of non-response components are given.     

Generalizability of valuations on health states collected with the EuroQolc-questionnaire

Problem: Non-response and non-usable response were found in population surveys on valuation of health states. If non-response is selective regarding valuations, then generalization of the resulting values to the whole survey population is not permitted. This could limit the use of empirical utility values in resource allocation in health care. Methods: Response behaviour of a sample of 1400 from the Dutch general population to the mailed EuroQol^c-questionnaire was analyzed by four methods. I. Phoning resolute non-respondents; II. comparison of zip code characteristics of respondents and non-respondents (because individual data on background characteristics were not available for the non-respondents); III. analysis of response over time (wave-analysis); IV: comparison of background variables of successful (less than two valuations missing) and unsuccessful respondents, combined with analysis of the effect of these background variables on valuations. Results: No indications for selective non-response were found, although the phenomenon appeared hard to investigate. The successful response came from a slightly younger and better educated subsample. However, a general influence of age and educational level on valuations could not be shown. This finding is consistent with the literature. Conclusion: Although the existence of selective non-response cannot be excluded, its relevance can be considered to be small. This finding is encouraging for the use of empirical utility values in allocative decisions.