On modelling response propensity for dwelling unit (DU) level non-response adjustment in the Medical Expenditure Panel Survey (MEPS)

Non-response is a common problem in household sample surveys. The Medical Expenditure Panel Survey (MEPS), sponsored by the Agency for Healthcare Research and Quality (AHRQ), is a complex national probability sample survey. The survey is designed to produce annual national and regional estimates of health-care use, expenditures, sources of payment, and insurance coverage for the U.S. civilian non-institutionalized population. The MEPS sample is a sub-sample of respondents to the prior year's National Health Interview Survey (NHIS) conducted by the National Center for Health Statistics (NCHS). The MEPS, like most sample surveys, experiences unit, or total, non-response despite intensive efforts to maximize response rates. This paper summarizes research on comparing alternative approaches for modelling response propensity to compensate for dwelling unit (DU), i.e. household level non-response in the MEPS.Non-response in sample surveys is usually compensated for by some form of weighting adjustment to reduce the bias in survey estimates. To compensate for potential bias in survey estimates in the MEPS, two separate non-response adjustments are carried out. The first is an adjustment for DU level non-response at the round one interview to account for non-response among those households subsampled from NHIS for the MEPS. The second non-response adjustment is a person level adjustment to compensate for attrition across the five rounds of data collection. This paper deals only with the DU level non-response adjustment. Currently, the categorical search tree algorithm method, the chi-squared automatic interaction detector (CHAID), is used to model the response probability at the DU level and to create the non-response adjustment cells. In this study, we investigate an alternative approach, i.e. logistic regression to model the response probability. Main effects models and models with interaction terms are both evaluated. We further examine inclusion of the base weights as a covariate in the logistic models. We compare variability of weights of the two alternative response propensity approaches as well as direct use of propensity scores.The logistic regression approaches produce results similar to CHAID; however, using propensity scores from logistic models with interaction terms to form five classification groups for weight adjustment appears to perform best in terms of limiting variability and bias. Published in 2007 by John Wiley & Sons, Ltd.     

Postal survey on airborne occupational exposure and respiratory disorders in Norway: causes and consequences of non-response.

STUDY OBJECTIVE--The aim was to examine causes for non-response in a community survey, and how non-response influences prevalence estimates of some exposure and disease variables, and associations between the variables. DESIGN--This was a cross sectional questionnaire study with two reminder letters. The questionnaire asked for information on smoking habits, occupational airborne exposure and respiratory disorders. SETTING--A random sample of 4992 subjects from the general population aged 15-70 years of Hordaland County, Norway. MAIN RESULTS--The overall response rate was 90%, with a 63% response to the initial letter. The response rates to the first and second reminder letters were 56% and 36% respectively. In 20% of the non-respondents an uncompleted questionnaire was returned with cause for non-response; in two thirds of these the cause for non-response was that the subject was not resident at the mailing address. A home visit to a random sample of 50 urban non-respondents provided further information on 29 subjects. A wrong address at the Central Population Registry and the subject's feeling of lack of personal benefit from a postal survey were the major reasons for non-response. Smokers were late respondents and subjects with respiratory disorders tended to be early respondents. CONCLUSION--The main reasons for non-response were a wrong mailing address and a feeling of lack of personal benefit from responding. Using only the initial letter would have changed the estimated prevalence of smokers from 39% to 35%. Otherwise, the estimated prevalence of the exposure and disease variables as well as the associations between them were only slightly changed after including the respondents to the first and second reminder letters.     

Evaluating classification performance of biomarkers in two-phase case-control studies

Biomarkers are playing an increasingly important role in disease screening, early detection, and risk prediction. The two-phase case-control sampling study design is widely used for the evaluation of candidate biomarkers. The sampling probabilities for cases and controls in the second phase can often depend on other covariates (sampling strata). This biased sampling can lead to invalid inference on a biomarker's classification accuracy if not properly accounted for. In this paper, we adopt the idea of inverse probability weighting and develop inverse probability weighting–based estimators for various measures of a biomarker's classification performance, including the points on the receiver operating characteristics (ROCs) curve, the area under the ROC curve (area under the curve), and the partial area under the curve. In particular, we consider classification accuracy estimators using sampling weights estimated conditionally on sampling strata and further improve their efficiency through the use of estimated weights that additionally take into account the auxiliary variables available from the phase-one cohort. We develop asymptotic properties of the proposed estimators and provide analytical variance for making inference. Extensive simulation studies demonstrate excellent performance of the proposed weighted estimators, while the traditional empirical estimator can be severely biased. We also investigate the advantages in efficiency gain for estimating various classification accuracy estimators through the use of auxiliary variables in addition to sampling strata and apply the proposed method to examples from a renal artery stenosis study and a prostate cancer study.     

Population coverage and nonresponse bias in a large-scale human exposure study

We used estimates derived from screener variables of the National Human Exposure Assessment Survey (NHEXAS) Phase I field study in EPA Region V (one of three NHEXAS Phase I field studies) to examine biases resulting from survey nonresponse and/or incomplete population coverage inherent in the study design. For variables with population values obtainable from Census projections, the combined effect of nonresponse and coverage bias was tested for after each stage of nonresponse using design-based weights. For variables where population values were not available as Census projections, nonresponse bias was tested for after the screener stage of nonresponse using weights adjusted for screener nonresponse. Additional tests for bias were performed using final survey weights to evaluate the performance of survey weight adjustments in reducing observed bias. Comparison of biases estimated using both design-based and adjusted weights was used to identify potentially important weight adjustment variables for future exposure studies, identify possible weaknesses in survey design strategies, and support the use of nonresponse and poststratification weight adjustments to reduce bias in future survey studies.     

Estimating disease prevalence from two-phase surveys with non-response at the second phase

In this paper we compare several methods for estimating population disease prevalence from data collected by two-phase sampling when there is non-response at the second phase. The traditional weighting type estimator requires the missing completely at random assumption and may yield biased estimates if the assumption does not hold. We review two approaches and propose one new approach to adjust for non-response assuming that the non-response depends on a set of covariates collected at the first phase: an adjusted weighting type estimator using estimated response probability from a response model; a modelling type estimator using predicted disease probability from a disease model; and a regression type estimator combining the adjusted weighting type estimator and the modelling type estimator. These estimators are illustrated using data from an Alzheimer's disease study in two populations.     

Association between variables used in the field substitution and post-stratification adjustment in the Belgian health interview survey and non-response

Objectives

Field substitution and post-stratification adjustment have been proposed to reduce non-response bias in population surveys. We investigated if variables involved in those techniques in the Belgian health interview survey 2004 are associated with non-response and assessed the impact of field substitution and post-stratification adjustment on the survey results.

Methods

Data were obtained from all selected households (n = 12.204). The association between non-response and the selected variables was explored through multilevel logistic regression models with municipality and statistical sector as random effects.

Results

All investigated variables were significantly related with non-response. Especially households that could not be contacted differed substantially from those who participated. Only post-stratification had a clear impact on the survey results.

Conclusions

Even if variables used in the field substitution procedure of health surveys are strongly associated with non-response, the impact of field substitution on the survey results may be minimal, either because there was no bias of relevance or it was not captured. The usefulness of field substitution to correct for non-response bias in population health surveys seems to be quite limited.     

Implementation and applications of bootstrap methods for the National Immunization Survey

In complex probability sample surveys, numerous adjustments are customarily made to the survey weights to reduce potential bias in survey estimates. These adjustments include sampling design (SD) weight adjustments, which account for features of the sampling plan, and non-sampling design (NSD) weight adjustments, which account for non-sampling errors and other effects. Variance estimates prepared from complex survey data customarily account for SD weight adjustments, but rarely account for all NSD weight adjustments. As a result, variance estimates may be biased and standard confidence intervals may not achieve their nominal coverage levels. We describe the implementation of the bootstrap method to account for the SD and NSD weight adjustments for complex survey data. Using data from the National Immunization Survey (NIS), we illustrate the use of the bootstrap (i). for evaluating the use of standard confidence intervals that use Taylor series approximations to variance estimators that do not account for NSD weight adjustments, (ii). for obtaining confidence intervals for ranks estimated from weighted survey data, and (iii). for evaluating the predictive power of logistic regressions using receiver operating characteristic curve analyses that account for the SD and NSD adjustments made to the survey weights.     

Characteristics of non-response in the Danish Health Interview Surveys, 1987-1994

BACKGROUND: The types and quantity of non-response in surveys influence the extent to which the results may be generalized. This study analysed trends in non-response in the Danish Health Interview Surveys from 1987 to 1994 and used the National Patient Registry to assess whether non-response biased the estimated population prevalence of morbidity when solely based on responders. METHODS: The data were for the 23,096 adults sampled for the Danish Health Interview Surveys in 1987, 1991 and 1994. All were followed using the National Patient Registry to obtain such information as hospital admissions. RESULTS: Non-response increased from 20.0% in 1987 to 22.6% in 1994. Four combinations of background variables characterized the non-response: gender and age; gender and civil status; county of residence and age; survey year and age. Non-respondents and respondents had identical gender- and age-standardized hospital admission rates for approximately 5 years before and 2 years after data collection, but non-respondents had a significantly higher rate immediately before and during data collection. Admissions rates were analysed according to reasons for non-response. Refusers had a lower admission rate than respondents before data collection but similar during and after data collection. The rate was higher during the whole period among ill or disabled non-respondents. Among people who could not be contacted during the data collection period a higher admission rate was only found immediately before and during data collection. CONCLUSIONS: Although admission rates differed between respondents and non-respondents these differences were too small to bias the estimated population prevalence of morbidity when solely based on respondents.     

Probability estimation when some observations are grouped

This paper considers the use of additional questions for decreasing survey non-response rates and an approach for estimating a probability based on the results obtained. In a survey, the respondents are asked to answer an original question and follow-up questions, where the answers for the follow-up questions are grouped answers for the original question. For example, respondents are asked to provide an exact number of incidents, but in cases of 'Do not know' or 'Refuse' responses, they are subsequently asked to pick an answer from a less specific categorical scale. The new estimator obtains smaller variance asymptotically and does not depend on a distribution family. This method is applied to income questions in a survey regarding injury prevention and behaviours. Another application is survey data on intimate partner violence, where some amendments were applied for incorporating post-stratification weights and for using non-random grouping. For additional illustration, an example of parameter estimation on artificially generated data is presented.     

Causal analysis of case-control data

In a series of papers, Robins and colleagues describe inverse probability of treatment weighted (IPTW) estimation in marginal structural models (MSMs), a method of causal analysis of longitudinal data based on counterfactual principles. This family of statistical techniques is similar in concept to weighting of survey data, except that the weights are estimated using study data rather than defined so as to reflect sampling design and post-stratification to an external population. Several decades ago Miettinen described an elementary method of causal analysis of case-control data based on indirect standardization. In this paper we extend the Miettinen approach using ideas closely related to IPTW estimation in MSMs. The technique is illustrated using data from a case-control study of oral contraceptives and myocardial infarction.     

Calibration weighted estimation of semiparametric transformation models for two‐phase sampling

Two‐phase designs are commonly used to subsample subjects from a cohort in order to study covariates that are too expensive to ascertain for everyone in the cohort. This is particularly true for the study of immune response biomarkers in vaccine immunology, where new, elaborate assays are constantly being developed to improve our understanding of the human immune responses to vaccines and how the immune response may protect humans from virus infection. It has long being recognized that if there exist variables that are correlated with expensive variables and can be measured for every subject in the cohort, they can be leveraged to improve the estimation efficiency for the effects of the expensive variables. In this research article, we developed an improved inverse probability weighted estimation approach for semiparametric transformation models with a two‐phase study design. Semiparametric transformation models are a class of models that include the Cox PH and proportional odds models. They provide an attractive way to model the effects of immune response biomarkers as human immune responses generally wane over time. Our approach is based on weights calibration, which has its origin in survey statistics and was used by Breslow et al. 1 , 2 to improve inverse probability weighted estimation of the Cox regression model. We develop asymptotic theory for our estimator and examine its performance through simulation studies. We illustrate the proposed method with application to two HIV‐1 vaccine efficacy trials. Copyright © 2015 John Wiley & Sons, Ltd.     

The effect of non-random loss to follow-up on group mean estimates in a longitudinal study

Bias due to selective non-response is often neglected in large-scale epidemiological studies. And, although some recent techniques enable adjustment for selective non-response, these are rarely applied. The Maastricht Cohort Study, a study on fatigue at work among 12140 respondents at baseline, enabled us to estimate the degree of bias in a real life data set. After seven subsequent measurements, spanning a 2-year period, 8070 respondents remained in the cohort. Two traditional ways of presenting longitudinal mean levels (means using all data, and means using only complete cases) are compared with adjusted mean levels, using mixed models. The difference between the complete case and overall mean levels and the adjusted means were about 2% for the continuous fatigue score and 6% for the proportion of fatigued cases. For the company mean scores the observed bias due to selective non-response might be as much as 30% for some of the company means for the continuous fatigue score and up to 160% for the estimated number of fatigued cases. We therefore conclude that bias due to selective non-response needs serious attention. Next to making vigorous attempts to minimize longitudinal non-response, the use of statistical adjustment is also recommended.     

死因监测整群抽样与不等概率抽样设计方案的比较

为比较不同整群抽样设计方法 的抽样误差及设计效应,评价不等概率抽样在死因监测中的应用效果.以陕西省107个县(市、区)作为抽样框架,采用等概率整群抽样和不等概率整群抽样等设计方案抽取样本,用复杂抽样方法 计算不同方案样本的抽样误差和设计效应.不同的抽样方案得到不同的抽样误差估计,分层整群抽样的标准误小于完全随机整群抽样;不等概率抽样(πPS抽样)的设计效率虽略逊于等概率的完全随机整群抽样,但扩大了监测范围.结论 :对于抽样框架明确的整群抽样调查数据,在统计分析时不应脱离预先设定的抽样设计方案和设计参数.死因监测采用不等概率抽样设计,能增加样本的权重,提高死亡率的地区代表性.     

Performance of binary markers for censored failure time outcome: nonparametric approach based on proportions

This work focuses on the assessment of the discrimination ability of a binary marker to identify patients that will relapse in time. We consider the cumulative definition of sensitivity and dynamic definition of specificity at a time horizon, that is, the probability of a positive marker in the population that will relapse (cases) and that will not relapse (controls). In the presence of censoring, sensitivity and specificity cannot be estimated by proportions because it is not known whether censored subjects should be considered as cases or controls. The solutions proposed do not enable to obtain asymptotic confidence intervals. We explore the use of inverse probability of censoring weighting/imputation (borrowed from the methodology used to correct for verification bias) to adjust the classification matrix for the presence of censoring. The adjustment based on weights estimated conditional on the marker turned to be equivalent to the adjustment based on imputation. These approaches, which address for the presence of marker-dependent censoring, showed a better performance than the adjustment based on weights estimated on the entire sample, even in the case of marker-independent censoring. We derived single intervals and confidence region for sensitivity and 1-specificity using the delta method. The confidence region is particularly useful for a binary marker because the marker has some ability to discriminate among cases and controls only if the region does not intersect the first quadrant bisector.     

Generalizability of valuations on health states collected with the EuroQolc-questionnaire

Problem: Non-response and non-usable response were found in population surveys on valuation of health states. If non-response is selective regarding valuations, then generalization of the resulting values to the whole survey population is not permitted. This could limit the use of empirical utility values in resource allocation in health care. Methods: Response behaviour of a sample of 1400 from the Dutch general population to the mailed EuroQol^c-questionnaire was analyzed by four methods. I. Phoning resolute non-respondents; II. comparison of zip code characteristics of respondents and non-respondents (because individual data on background characteristics were not available for the non-respondents); III. analysis of response over time (wave-analysis); IV: comparison of background variables of successful (less than two valuations missing) and unsuccessful respondents, combined with analysis of the effect of these background variables on valuations. Results: No indications for selective non-response were found, although the phenomenon appeared hard to investigate. The successful response came from a slightly younger and better educated subsample. However, a general influence of age and educational level on valuations could not be shown. This finding is consistent with the literature. Conclusion: Although the existence of selective non-response cannot be excluded, its relevance can be considered to be small. This finding is encouraging for the use of empirical utility values in allocative decisions.     

Non-response and related factors in a nation-wide health survey

Objective: To analyse selective factors associated with an unexpectedly low response rate. Subjects and methods: The baseline questionnaire survey of a large prospective follow-up study on the psychosocial health of the Finnish working-aged randomly chosen population resulted in 21,101 responses (40.0%) in 1998. The non-respondent analysis used demographic and health-related population characteristics from the official statistics and behavioural, physical and mental health-related outcome differences between early and late respondents to predict possible non-response bias. Reasons for non-response, indicated by missing responses of late respondents, and factors affecting the giving of consent were also analysed. Results: The probability of not responding was greater for men, older age groups, those with less education, divorced and widowed respondents, and respondents on disability pension. The physical health-related differences between the respondents and the general population were small and could be explained by differences in definitions. The late respondents smoked and used more psychopharmaceutical drugs than the early ones, suggesting similar features in non-respondents. The sensitive issues had a small effect on the response rate. The consent to use a medical register-based follow-up was obtained from 94.5% of the early and 90.9% of the late respondents (odds ratio: 1.70; 95% confidence interval: 1.49–1.93). Consent was more likely among respondents reporting current smoking, heavy alcohol use, panic disorder or use of tranquillisers. Conclusions: The main reasons for non-response may be the predisposing sociodemographic and behavioural factors, the length and sensitive nature of the questionnaire to some extent, and a suspicion of written consent and a connection being made between the individual and the registers mentioned on the consent form.     

Effect on Trend Estimates of the Difference between Survey Respondents and Non-respondents: Results from 27 Populations in the WHO MONICA Project

Introduction: In the World Health Organization (WHO) MONICA (multinational MONItoring of trends and determinants in CArdiovascular disease) Project considerable effort was made to obtain basic data on non-respondents to community based surveys of cardiovascular risk factors. The first purpose of this paper is to examine differences in socio-economic and health profiles among respondents and non-respondents. The second purpose is to investigate the effect of non-response on estimates of trends. Methods:Socio-economic and health profile between respondents and non-respondents in the WHO MONICA Project final survey were compared. The potential effect of non-response on the trend estimates between the initial survey and final survey approximately ten years later was investigated using both MONICA data and hypothetical data. Results: In most of the populations, non-respondents were more likely to be single, less well educated, and had poorer lifestyles and health profiles than respondents. As an example of the consequences, temporal trends in prevalence of daily smokers are shown to be overestimated in most populations if they were based only on data from respondents. Conclusions: The socio-economic and health profiles of respondents and non-respondents differed fairly consistently across 27 populations. Hence, the estimators of population trends based on respondent data are likely to be biased. Declining response rates therefore pose a threat to the accuracy of estimates of risk factor trends in many countries.     

Equivalence of Mail and Telephone Responses to the CAHPS® Hospital Survey

Objective. To estimate the effect of survey mode (mail versus telephone) on reports and ratings of hospital care.
Data Sources/Study Setting. The total sample included 20,826 patients discharged from a group of 24 distinct hospitals in three states (Arizona, Maryland, New York). We collected CAHPS^® data in 2003 by mail and telephone from 9,504 patients, of whom 39 percent responded by telephone and 61 percent by mail.
Study Design. We estimated mode effects in an observational design, using both propensity score blocking and (ordered) logistic regression on covariates. We used variables derived from administrative data (either included as covariates in the regression function or used in estimating the propensity score) grouped in three categories: individual characteristics, characteristics of the stay and hospital, and survey administration variables.
Data Collection/Extraction Methods. We mailed a 66-item questionnaire to everyone in the sample and followed up by telephone with those who did not respond.
Principal Findings. We found significant ( p <.01) mode effects for 13 of the 21 questions examined in this study. The maximum magnitude of the survey mode effect was an 11 percentage-point difference in the probability of a "yes" response to one of the survey questions. Telephone respondents were more likely to rate care positively and health status negatively, compared with mail respondents. Standard regression-based case-mix adjustment captured much of the mode effects detected by propensity score techniques in this application.
Conclusions. Telephone mode increases the propensity for more favorable evaluations of care for more than half of the items examined. This suggests that mode of administration should be standardized or carefully adjusted for. Alternatively, further item development may minimize the sensitivity of items to mode of data collection.     

A Bayesian analysis of a proportion under non-ignorable non-response

The National Health Interview Survey (NHIS) is one of the surveys used to assess one aspect of the health status of the U.S. population. One indicator of the nation's health is the total number of doctor visits made by the household members in the past year. We study the binary variable of at least one doctor visit versus no doctor visit by all household members to each of the 50 states and the District of Columbia. The proportion of households with at least one doctor visit is an indicator of the status of health of the U.S. population. There is a substantial number of non-respondents among the sampled households. The main issue we address here is that the non-response mechanism should not be ignored because respondents and non-respondents differ. The purpose of this work is to estimate the proportion of households with at least one doctor visit, and to investigate what adjustment needs to be made for non-ignorable non-response. We consider a non-ignorable non-response model that expresses uncertainty about ignorability through the ratio of odds of a household doctor visit among respondents to the odds of doctor visit among all households, and this ratio varies from state to state. We use a hierarchical Bayesian selection model to accommodate this non-response mechanism. Because of the weak identifiability of the parameters, it is necessary to 'borrow strength' across states as in small area estimation. We also perform a simulation study to compare the expansion model with an alternative expansion model, an ignorable model and a non-ignorable model. Inference for the probability of a doctor visit is generally similar across the models. Our main result is that for some of the states the non-response mechanism can be considered non-ignorable, and that 95 per cent credible intervals of the probability for a household doctor visit and the probability that a household responds shed important light on the NHIS data.