Effect of sampling frames on response rates in the WHO MONICA risk factor surveys

Sample surveys are used to investigate occurrence and determinants of diseases in populations. Their reliability is influenced by quality of sampling frame and response rate. We investigated relationship between sampling frame type and response rates and assessed their impact on non-response bias, using data from the WHO MONICA Project, where 37 centres in 20 countries conducted sample surveys, employing the best locally available sampling frame. Sampling frames fell into three categories: Population registers (PR), electoral registers (ER), and health care registers (HR). Response rate (rrs) was factored into components reflecting quality of sampling frame (contact rate cr) and characterizing willingness of sample members to participate (enrolment rate er). The mean quality score for the sampling frames was 92 for PR, 87 for HR and 85 for ER; they contributed on average 23, 20, and 26 to the respective non-response rates. For all frame types and both sexes the lowest quality score occurred in the age group 35–44, suggesting a reduced ability to track migration of a highly mobile population group. The patterns in the age/sex distribution of er indicate at least for males in PR and females in HR a potential for non-response bias. Estimation of non-response bias through an abbreviated questionnaire failed because of low item response. We found that contact rate characterizes sampling frame quality. For all frame types it had a major influence on response rate. It is likely that low er and low cr cause different kind of bias, requiring different measures to minimize their effects.for the WHO MONICA Project** Sites and key personnel of the WHO MONICA Project are found at http://www.ktl.fi/publications/monica/rr_sframe/appendix.htm     

Generalizability of valuations on health states collected with the EuroQolc-questionnaire

Problem: Non-response and non-usable response were found in population surveys on valuation of health states. If non-response is selective regarding valuations, then generalization of the resulting values to the whole survey population is not permitted. This could limit the use of empirical utility values in resource allocation in health care. Methods: Response behaviour of a sample of 1400 from the Dutch general population to the mailed EuroQol^c-questionnaire was analyzed by four methods. I. Phoning resolute non-respondents; II. comparison of zip code characteristics of respondents and non-respondents (because individual data on background characteristics were not available for the non-respondents); III. analysis of response over time (wave-analysis); IV: comparison of background variables of successful (less than two valuations missing) and unsuccessful respondents, combined with analysis of the effect of these background variables on valuations. Results: No indications for selective non-response were found, although the phenomenon appeared hard to investigate. The successful response came from a slightly younger and better educated subsample. However, a general influence of age and educational level on valuations could not be shown. This finding is consistent with the literature. Conclusion: Although the existence of selective non-response cannot be excluded, its relevance can be considered to be small. This finding is encouraging for the use of empirical utility values in allocative decisions.     

Characteristics of non-response in the Danish Health Interview Surveys, 1987-1994

BACKGROUND: The types and quantity of non-response in surveys influence the extent to which the results may be generalized. This study analysed trends in non-response in the Danish Health Interview Surveys from 1987 to 1994 and used the National Patient Registry to assess whether non-response biased the estimated population prevalence of morbidity when solely based on responders. METHODS: The data were for the 23,096 adults sampled for the Danish Health Interview Surveys in 1987, 1991 and 1994. All were followed using the National Patient Registry to obtain such information as hospital admissions. RESULTS: Non-response increased from 20.0% in 1987 to 22.6% in 1994. Four combinations of background variables characterized the non-response: gender and age; gender and civil status; county of residence and age; survey year and age. Non-respondents and respondents had identical gender- and age-standardized hospital admission rates for approximately 5 years before and 2 years after data collection, but non-respondents had a significantly higher rate immediately before and during data collection. Admissions rates were analysed according to reasons for non-response. Refusers had a lower admission rate than respondents before data collection but similar during and after data collection. The rate was higher during the whole period among ill or disabled non-respondents. Among people who could not be contacted during the data collection period a higher admission rate was only found immediately before and during data collection. CONCLUSIONS: Although admission rates differed between respondents and non-respondents these differences were too small to bias the estimated population prevalence of morbidity when solely based on respondents.     

Survey non-response in the Netherlands: effects on prevalence estimates and associations

PURPOSE: Differences in respondent characteristics may lead to bias in prevalence estimates and bias in associations. Both forms of non-response bias are investigated in a study on psychosocial factors and cancer risk, which is a sub-study of a large-scale monitoring survey in the Netherlands. METHODS: Respondents of a cross-sectional monitoring project (MORGEN; N = 22,769) were also asked to participate in a prospective study on psychosocial factors and cancer risk (HLEQ; N = 12,097). To investigate diverse aspects of non-response in the HLEQ on prevalence estimates and associations are studied, based on information gathered in the MORGEN-project. RESULTS: A response percentage of 45% was obtained in the MORGEN-project. Response rates were found to be lower among men and younger people. The HLEQ showed a response percentage of 56%, and respondents reported higher socioeconomic status, better subjective health and healthier lifestyle behaviors than non-respondents. However, associations between smoking status and either socioeconomic status or subjective health based on respondents only were not statistically different from those based on the entire MORGEN-population. CONCLUSION: Non-response leads to bias in prevalence estimates of current smoking, current alcohol intake, and low physical activity or poor subjective health. However, non-response did not cause bias in the examined associations.     

The effect of non-response on estimates of health care utilisation: linking health surveys and registers

BACKGROUND: Non-response in health surveys may lead to bias in estimates of health care utilisation. The magnitude, direction and composition of the bias are usually not well known. When data from health surveys are merged with data from registers at the individual level, analyses can reveal non-response bias. Our aim was to estimate the composition, direction and magnitude of non-response bias in the estimation of health care costs in two types of health interview surveys. METHODS: The surveys were (1) a national personal interview survey of 22 484 Danes (2) a telephone interview survey of 5000 Danes living in Funen County. Data were linked with register information on health care utilisation in hospitals and primary care. Health care utilisation was estimated for respondents and non-respondents, and the difference was explained by a decomposition method of bias components. RESULTS: The surveys produced the same pattern of non-response, but with slight differences in non-response bias. Response rates for the interview and telephone surveys were 75 and 69%, respectively. Refusal was the most frequent reason for non-response (22 and 20% of those sampled, respectively), whereas illness, non-contact, and other reasons were less frequent. Respondents used 3-6% less health care than non-respondents at the aggregate level, but the opposite was true for some specific types of health care. Non-response due to illness was the main contributor to non-response bias. CONCLUSIONS: Different types of non-response have different bias effects. However, the magnitude of the bias encourages the continued use of interview health surveys.     

Does survey non-response bias the association between occupational social class and health?

AIMS: A non-response rate of 20-40%is typical in questionnaire studies. The authors evaluate non-response bias and its impact on analyses of social class inequalities in health. METHODS: Set in the context of a health survey carried out among the employees of the City of Helsinki (non-response 33%) in 2000-02. Survey response and non-response records were linked with a personnel register to provide information on occupational social class and long sickness absence spells as an indicator of health status. RESULTS: Women and employees in higher occupational social classes were more likely to respond. Non-respondents had about 20-30% higher sickness absence rates. Relative social class differences in sickness absence in the total population were similar to those among either respondents or non-respondents. CONCLUSIONS: In working populations survey non-response does not seriously bias analyses of social class inequalities in sickness absence and possibly health inequalities more generally.     

Medical consumption of respondents and non-respondents to a mailed health survey

Non-response bias can distort the results of health surveys.The occurrence of selective non-response can be assessed whendata are available for both respondents and non-respondents.The objective of this study was to compare the medical consumptionof respondents and non-respondents to a mailed health survey.A mailed health survey was conducted among approximately 13,500adults and among parents of approximately 1,500 children aged5–15 years. The net response rate was 70.4%. A panel dataset that could be matched with the health survey data was availablefor all eligible persons. This data set comprises administrativeinformation on hospitalizations, annual health care expendituresand demographic variables. The results of this study show thatresponse was associated with age, sex, degree of urbanizationand type of insurance. After correcting for differences in demographicvariables, respondents and non-respondents differ in the utilizationof several types of care. Relatively more users than non-usersresponded. Response was not associated with the utilizationof care related to severe conditions such as in-patient hospitalcare. The conclusion from this study is that when a mailed healthsurvey is used to measure medical consumption, the non-responsebias will result in a small overestimation of utilization.     

Marital status,educational level and household income explain part of the excess mortality of survey non-respondents

Survey respondents and non-respondents differ in their demographic and socio-economic position. Many of the health behaviours are also known to be associated with socio-economic differences. We aimed to investigate how much of the excess mortality of survey non-respondents can be explained by the socio-economic differences between respondents and non-respondents. Questionnaire-based adult health behaviour surveys have been conducted in Finland annually since 1978. Data from the 1978 to 2002 surveys, including non-respondents, were linked with mortality data from the Finnish National Cause of Death statistics and with demographic and socio-economic register data (marital status, education and household income) obtained from Statistics Finland. The mortality follow-up lasted until 2006, in which period there were 12,762 deaths (7,994 in men and 4,768 in women) during the follow-up. Total and cause-specific mortality were higher among non-respondents in both men and women. Adjusting results for marital status, educational level and average household income decreased the excess total and cause-specific mortality of non-respondents in both men and women. Of the total excess mortality of non-respondents, 41% in men and 20% in women can be accounted for demographic and socio-economic factors. A part of the excess mortality among non-respondents can be accounted for their demographic and socio-economic characteristics. Based on these results we can assume that non-respondents tend to have more severe health problems, acute illnesses and unhealthy behaviours, such as smoking and excess alcohol use. These can be reasons for persons not taking part in population surveys.     

Selection bias due to non-response in a health survey among patients with rheumatoid arthritis

BACKGROUND: Non-response may lead to bias in health(care) outcomes. METHODS: We compared respondents (n = 334) to a questionnaire survey among patients with rheumatoid arthritis with non-respondents (n = 68) and determined predictors of (non-)response. The bias in prevalence estimates of health characteristics and health care use was quantified. RESULTS: Self-reported pain and health care utilization were the most important predictors of (non-)response with respondents experiencing pain more often and more often using specific health care services. Bias concerned especially an underestimation of 'never having pain' (60%) and 'no contact with health care services' (51%). CONCLUSION: More insight into the phenomenon of non-response is important to assess disease burden and health care burden more precisely.     

Non-response and related factors in a nation-wide health survey

Objective: To analyse selective factors associated with an unexpectedly low response rate. Subjects and methods: The baseline questionnaire survey of a large prospective follow-up study on the psychosocial health of the Finnish working-aged randomly chosen population resulted in 21,101 responses (40.0%) in 1998. The non-respondent analysis used demographic and health-related population characteristics from the official statistics and behavioural, physical and mental health-related outcome differences between early and late respondents to predict possible non-response bias. Reasons for non-response, indicated by missing responses of late respondents, and factors affecting the giving of consent were also analysed. Results: The probability of not responding was greater for men, older age groups, those with less education, divorced and widowed respondents, and respondents on disability pension. The physical health-related differences between the respondents and the general population were small and could be explained by differences in definitions. The late respondents smoked and used more psychopharmaceutical drugs than the early ones, suggesting similar features in non-respondents. The sensitive issues had a small effect on the response rate. The consent to use a medical register-based follow-up was obtained from 94.5% of the early and 90.9% of the late respondents (odds ratio: 1.70; 95% confidence interval: 1.49–1.93). Consent was more likely among respondents reporting current smoking, heavy alcohol use, panic disorder or use of tranquillisers. Conclusions: The main reasons for non-response may be the predisposing sociodemographic and behavioural factors, the length and sensitive nature of the questionnaire to some extent, and a suspicion of written consent and a connection being made between the individual and the registers mentioned on the consent form.     

Characteristics of respondents and non-respondents from a case-control study of breast cancer in younger women

BACKGROUND: This study assessed the nature of potential biases by comparing respondents with non-respondents from a case-control study of breast cancer in younger women. METHODS: The case-control study was conducted in three regions in the US: Atlanta GA, Seattle/Puget Sound WA, and central New Jersey. An abbreviated interview or mailed questionnaire was completed by willing non-respondents, most of whom had refused participation in the main study. RESULTS: Respondents and non-respondents appeared similar with respect to age, race, relative weight, smoking, family history of breast cancer, number of births, age at first birth, and several dietary items. Compared to non-respondents, case and control respondents were of shorter stature, and reported less frequent consumption of doughnuts/pastries. Respondent cases, compared with non-respondent cases, were more highly educated and more likely to have consumed alcohol regularly; similar but not statistically significant tendencies were observed for controls. Respondent cases experienced menarche earlier than non-respondents. Respondent controls were more likely to have used oral contraceptives than non-respondents; a similar but not statistically significant tendency was observed in cases. Comparisons of crude and simulated relative risks using available non-respondents' data generally showed a low impact of non-response on relative risks in this study. CONCLUSIONS: Our results suggest that non-response would not greatly affect relative risk estimates in this study, except possibly regarding height. However, we were limited by the numbers of informative non-respondents and the amount of data collected. Collecting similar information in future studies would be useful, especially since varying methods used to encourage participation may lead to differences in respondents' characteristics.     

Predicting risk among non-respondents in prospective studies

Potential non-response bias was investigated in a follow-up study of 2,011 chronically disabled patients. 82.5% and 73.3% of the study subjects responded to self-administered mail questionnaires respectively at 6-month and 1-year follow-up. Information on employment status, the outcome of interest, of approximately 90% of the non-respondents was obtained from indirect sources. Employment rate was lower among the non-respondents than the respondents. Non-response was associated with age, social class, previous employment record, and the type of disability; but none of these characteristics were associated with the outcome. Out of the five known independent risk factors for unemployment, only one (incompletion of rehabilitation course) was associated with non-response. The employment rate among the respondents was also assessed according to the delay in response, that is the number of reminders sent to achieve response. The outcome among- the late respondents was similar to that among the nonrespondents. These data suggest that (a) risk estimates may be biased even when the response rate is greater than 80%, (b) the prevalence of risk factors among non-respondents may not indicate the presence or the degree of non-response bias, but (c) reliable estimates can be obtained from extrapolations of the rates among the respondents according to the delay in response.     

Adolescent survey non-response and later risk of death. A prospective cohort study of 78 609 persons with 11-year follow-up

Background  

Non-response in survey studies is a growing problem and, being usually selective, it leads to under- or overestimation of health outcomes in the follow-up. We followed both respondents and non-respondents by registry linkage to determine whether there is a risk of death, related to non-response at baseline.     

Trends in cigarette smoking in 36 populations from the early 1980s to the mid-1990s: findings from the WHO MONICA Project

OBJECTIVES: This report analyzes cigarette smoking over 10 years in populations in the World Health Organization (WHO) MONICA Project (to monitor trends and determinants of cardiovascular disease). METHODS: Over 300,000 randomly selected subjects aged 25 to 64 years participated in surveys conducted in geographically defined populations. RESULTS: For men, smoking prevalence decreased by more than 5% in 16 of the 36 study populations, remained static in most others, but increased in Beijing. Where prevalence decreased, this was largely due to higher proportions of never smokers in the younger age groups rather than to smokers quitting. Among women, smoking prevalence increased by more than 5% in 6 populations and decreased by more than 5% in 9 populations. For women, smoking tended to increase in populations with low prevalence and decrease in populations with higher prevalence; for men, the reverse pattern was observed. CONCLUSIONS: These data illustrate the evolution of the smoking epidemic in populations and provide the basis for targeted public health interventions to support the WHO priority for tobacco control.     

Postal survey on airborne occupational exposure and respiratory disorders in Norway: causes and consequences of non-response.

STUDY OBJECTIVE--The aim was to examine causes for non-response in a community survey, and how non-response influences prevalence estimates of some exposure and disease variables, and associations between the variables. DESIGN--This was a cross sectional questionnaire study with two reminder letters. The questionnaire asked for information on smoking habits, occupational airborne exposure and respiratory disorders. SETTING--A random sample of 4992 subjects from the general population aged 15-70 years of Hordaland County, Norway. MAIN RESULTS--The overall response rate was 90%, with a 63% response to the initial letter. The response rates to the first and second reminder letters were 56% and 36% respectively. In 20% of the non-respondents an uncompleted questionnaire was returned with cause for non-response; in two thirds of these the cause for non-response was that the subject was not resident at the mailing address. A home visit to a random sample of 50 urban non-respondents provided further information on 29 subjects. A wrong address at the Central Population Registry and the subject's feeling of lack of personal benefit from a postal survey were the major reasons for non-response. Smokers were late respondents and subjects with respiratory disorders tended to be early respondents. CONCLUSION--The main reasons for non-response were a wrong mailing address and a feeling of lack of personal benefit from responding. Using only the initial letter would have changed the estimated prevalence of smokers from 39% to 35%. Otherwise, the estimated prevalence of the exposure and disease variables as well as the associations between them were only slightly changed after including the respondents to the first and second reminder letters.     

Estimation of non-response bias in the Medicare FFS HOS

We examined non-response bias in physical component summary scores (PCS) and mental component summary scores (MCS) in the Medicare fee-for-service (FFS) Health Outcomes Survey (HOS) using two alternative methods, response propensity weighting and imputation for non-respondents. The two approaches gave nearly identical estimates of non-response bias. PCS scores were 0.74 points lower and MCS scores 0.51 points lower after adjustment for non-response through imputation and 0.63 and 0.46 lower after adjustment for propensity weighting. These levels are small for component scores suggesting that survey non-response to the FFS HOS does not adversely affect estimates of average health status for this population.     

Estimating non-response bias in a survey on alcohol consumption: comparison of response waves

AIMS: According to 'the continuum of resistance model' late respondents can be used as a proxy for non-respondents in estimating non-response bias. In the present study, the validity of this model was explored and tested in three surveys on alcohol consumption. METHODS: The three studies collected their data by means of mailed questionnaires on alcohol consumption whereby two studies also performed a non-response follow-up. RESULTS: Comparisons of early respondents, late respondents and non-respondents in one study showed some support for 'the continuum of resistance model', although another study could not confirm this result. Comparison of alcohol consumption between three time response groups showed no significant linear pattern of differences between response waves. CONCLUSIONS: The hypothesis that late respondents are more similar to non-respondents than early respondents, could not be confirmed or rejected. Repeated mailings are effective in obtaining a greater sample size, but seem ineffective in improving the representativeness of alcohol consumption surveys.     

Recruitment methods and differences in early, late and non-respondents in the first MONICA-Catalonia population survey

The early response rate in the first MONICA-Catalonia population survey was 52.7% and the final response rate was 73.8%. The intensity of recruitment effort in this survey led to a considerable increase in response rate (20%), with the extra cost per late respondent being relatively low ($13.9). Added recruitment effort was most effective in the youngest age group, 25-34 years. It was also more effective among women living in urban areas than among those from rural areas. In men, early respondents had a higher proportion of smokers than late respondents, and in women, early respondents had higher systolic and diastolic blood pressure levels and were more aware of their history of high blood pressure than late respondents. Non-respondents were less educated than respondents in both sexes, and this was more marked in women. No differences were found in the proportion of smokers between respondents and non-respondents. Respondents were more aware of their high blood pressure history than non-respondents. The recruitment costs and distribution of non-response components are given.     

Educational level, relative body weight, and changes in their association over 10 years: an international perspective from the WHO MONICA Project

OBJECTIVES: This study assessed the consistency and magnitude of the association between educational level and relative body weight in populations with widely different prevalences of over-weight and investigated possible changes in the association over 10 years. METHODS: Differences in age-adjusted mean body mass index (BMI) between the highest and the lowest tertiles of years of schooling were calculated for 26 populations in the initial and final surveys of the World Health Organization (WHO) MONICA (Monitoring Trends and Determinants in Cardiovascular Disease) Project. The data are derived from random population samples, including more than 42,000 men and women aged 35 to 64 years in the initial survey (1979-1989) and almost 35,000 in the final survey (1989-1996). RESULTS: For women, almost all populations showed a statistically significant inverse association between educational level and BMI; the difference between the highest and the lowest educational tertiles ranged from -3.3 to 0.4 kg/m2. For men, the difference ranged from -1.5 to 2.2 kg/m2. In about two thirds of the populations, the differences in BMI between the educational levels increased over the 10-year period. CONCLUSION: Lower education was associated with higher BMI in about half of the male and in almost all of the female populations, and the differences in relative body weight between educational levels increased over the study period. Thus, socioeconomic inequality in health consequences of obesity may increase in many countries.