Commentary: Trade-offs in the development of a sample design for case-control studies

The recent article, "Comparison of Telephone Sampling and Area Sampling: Response Rates and Within-Household Coverage" (Am J Epidemiol 2001;153:1119-27), raised a number of issues related to two sampling methodologies that can be used for selecting population-based controls for case-control studies: random digit dialing (RDD) and area probability sampling. Some of these issues are discussed in this commentary in more detail to help in making sample design decisions, including the need to take the analysis plan into account when developing a sample design. Data from the paper are used to illustrate how the choice of sample design can affect analyses. Relative costs associated with the two methodologies as well as variance and bias concerns are also discussed in detail. Sample coverage issues, including those associated with list-assisted RDD, are considered, as are some advantages of the list-assisted approach. A discussion of the use of concurrent screening and sampling with an RDD approach as an alternative to periodically selecting fixed sample sizes is provided.     

A comparison of different matching designs in case-control studies: An empirical example using continuous exposures,continuous confounders and incidence of myocardial infarction

The paper presents a case-control study involving a disease, exposures and several continuous confounders. The relative efficiency and validity of a fully matched design is compared with random sampling of controls. We test a viable option of a partially matched design when inability to match all study subjects on all confounders occurs. The degree of bias in the odds ratios introduced by the different designs and by the different analytic models is assessed in comparison with the estimates obtained from a total cohort, from which both cases and controls were selected. Matched designs and analytic strategies are also evaluated in terms of the variances of the odds ratios. The results indicate that matching on continuous variables may lead to a more precise estimate of odds ratio than statistical control of confounding in unmatched designs. Partial selection of controls by matching may be a useful strategy when complete matching cannot be achieved; in practice, partial matching achieves most of the benefits of full matching.     

Random digit dialling and Electronic White Pages samples compared: demographic profiles and health estimates

OBJECTIVE: To compare the methodologies of and health estimates derived from two telephone household survey methods. In particular, to establish if White Pages telephone listings provide a relatively unbiased sampling frame for population health surveys. METHOD: In South Australia in 1998, a health survey questionnaire was administered by telephone to two randomly selected population samples. The first method used EWP (Electronic White Pages, n = 6,012), which contains all listed residential telephone numbers as the sampling frame. The results were compared to a RDD (random digit dialling, n = 3,080) sample where all listed and unlisted telephone numbers were included in the sampling frame. Demographic variables and health estimates were compared between the surveys, and then compared to a 'gold standard' door-to-door household survey conducted concurrently. RESULTS: The response rate for EWP (83.8%) exceeded that of RDD (65.4%). More than four times as many calls were required per completed interview in RDD. Demographic profiles and health estimates were substantially similar. CONCLUSIONS: EWP requires fewer telephone calls and enables approach letters establishing the bona fides of the survey to be sent to each selected address before calling, increasing the response rate. RDD is a more inclusive sampling frame but also includes non-connected and business numbers, and offers no significant advantages in providing health estimates. IMPLICATIONS: There are substantial methodological and cost advantages in using EWP over RDD as the sampling, frame for population health surveys, without introducing significant bias into health estimates.     

Selecting controls for assessing interaction in nested case-control studies

BACKGROUND: Two methods for selecting controls in nested case-control studies--matching on X and counter matching on X--are compared when interest is in interaction between a risk factor X measured in the full cohort and another risk factor Z measured only in the case-control sample. This is important because matching provides efficiency gains relative to random sampling when X is uncommon and the interaction is positive (greater than multiplicative), whereas counter matching is generally efficient compared to random sampling. METHODS: Matching and counter matching were compared to each other and to random sampling of controls for dichotomous X and Z. Comparison was by simulation, using as an example a published study of radiation and other risk factors for breast cancer in the Japanese atomic-bomb survivors, and by asymptotic relative efficiency calculations for a wide range of parameters specifying the prevalence of X and Z as well as the levels of correlation and interaction between them. Focus was on analyses utilizing general models for the joint risk of X and Z. RESULTS: Counter-matching performed better than matching or random sampling in terms of efficiency for inference about interaction in the case of a rare risk factor X and uncorrelated risk factor Z. Further, more general, efficiency calculations demonstrated that counter-matching is generally efficient relative to matched case-control designs for studying interaction. CONCLUSIONS: Because counter-matched designs may be analyzed using standard statistical methods and allow investigation of confounding of the effect of X, whereas matched designs require a non-standard approach when fitting general risk models and do not allow investigating the adjusted risk of X, it is concluded that counter-matching on X can be a superior alternative to matching on X in nested case-control studies of interaction when X is known at the time of case-control sampling.     

Experience with multiple control groups in a large population-based case–control study on genetic and environmental risk factors

We discuss the analytic and practical considerations in a large case–control study that had two control groups; the first control group consisting of partners of patients and the second obtained by random digit dialling (RDD). As an example of the evaluation of a general lifestyle factor, we present body mass index (BMI). Both control groups had lower BMIs than the patients. The distribution in the partner controls was closer to that of the patients, likely due to similar lifestyles. A statistical approach was used to pool the results of both analyses, wherein partners were analyzed with a matched analysis, while RDDs were analyzed without matching. Even with a matched analysis, the odds ratio with partner controls remained closer to unity than with RDD controls, which is probably due to unmeasured confounders in the comparison with the random controls as well as intermediary factors. However, when studying injuries as a risk factor, the odds ratio remained higher with partner control subjects than with RRD control subjects, even after taking the matching into account. Finally we used factor V Leiden as an example of a genetic risk factor. The frequencies of factor V Leiden were identical in both control groups, indicating that for the analyses of this genetic risk factor the two control groups could be combined in a single unmatched analysis. In conclusion, the effect measures with the two control groups were in the same direction, and of the same order of magnitude. Moreover, it was not always the same control group that produced the higher or lower estimates, and a matched analysis did not remedy the differences. Our experience with the intricacies of dealing with two control groups may be useful to others when thinking about an optimal research design or the best statistical approach.     

The NICHD Women's Contraceptive and Reproductive Experiences Study: methods and operational results

PURPOSE: This paper presents methods and operational results of a population-based case-control study examining the effects of oral contraceptive use on breast cancer risk among white and black women aged 35-64 years in five U.S. locations. METHODS: Cases were women newly diagnosed with breast cancer during July 1994 through April 1998. Controls were identified through random digit dialing (RDD) using unclustered sampling with automated elimination of nonworking numbers. Sampling was density-based, with oversampling of black women. In-person interviews were conducted from August 1994 through December 1998. Blood samples were obtained from subsets of cases and controls, and tissue samples were obtained from subsets of cases. A computerized system tracked subjects through study activities. Special attention was devoted to minimizing exposure misclassification, because any exposure-disease associations were expected to be small. RESULTS: An estimated 82% of households were screened successfully through RDD. Interviews were completed for 4575 cases (2953 whites; 1622 blacks) and 4682 controls (3021 whites; 1661 blacks). Interview response rates for cases and controls were 76.5% and 78.6%, respectively, with lower rates for black women and older women. CONCLUSIONS: The methodologic details of this large collaboration may assist researchers conducting similar investigations.     

Secular trends in response rates for controls selected by random digit dialing in childhood cancer studies: a report from the Children's Oncology Group

Since the mid-1990s, epidemiologists have anecdotally reported difficulty in recruiting controls using random digit dialing (RDD), but few empirical data have been published. From 1982 to 2003, epidemiologists from the Children's Oncology Group conducted 17 case-control studies using RDD controls. Data for calculating RDD and field response rates were available from eight and 13 of these studies, respectively. Over the period of analysis, the contact rate declined 2.5% per year (95% confidence interval (CI): -3.4, -1.6; p = 0.001), from above 90% in the 1980s to 63-69% in the most recent studies. The response rate (the product of the contact and cooperation rates) showed a decline parallel to that of the contact rate (-2.4% per year, 95% CI: -3.2, -1.6; p < 0.001), from above 80% in the 1980s to 50-67% after the mid-1990s. Field response rates appeared to have declined modestly. The overall response rate (the product of the RDD response and field response rates) paralleled that of the RDD response rate and decreased 2.4% per year (95% CI: -2.7, -2.0; p < 0.001). The current low response rates for RDD indicate a substantial potential for selection bias and a need to seek alternative sources of controls.     

Population- and community-based recruitment of African Americans and Latinos: the San Francisco Bay Area Lung Cancer Study

Empiric data on recruitment of minorities into clinical or population studies are limited. The authors evaluated population- and community-based recruitment methods in a 1998-2001 case-control study of lung cancer among African Americans and Latinos. For lung cancer cases in the San Francisco Bay Area of California, rapid case ascertainment by the tumor registry combined with telephone screening identified 470 (9%) African Americans and 262 (5%) Latinos. When random digit dialing (RDD) and Health Care Financing Administration (HCFA) records failed to yield adequate numbers of controls in appropriate age-gender-ethnicity groups, community-based recruitment methods were used. Demographic characteristics and behavioral and occupational risk factors for controls, by recruitment method, were compared with those for lung cancer cases to evaluate potential bias. The average numbers of hours spent per control recruited were 18.6 for RDD, 11.4 for HCFA, and less than 1 for the community-based methods. The prevalence of smoking-related lung cancer risk factors was significantly higher among African-American community-based controls than for those identified through RDD (p < 0.005). Compared with HCFA controls, Latino RDD controls reported significantly higher cumulative smoking exposure (p < 0.05). Further assessment of strategies for successful recruitment of minority participants into epidemiologic studies is warranted.     

Design and conduct of occupational epidemiology studies: III. Design aspects of case-control studies

Currently available approaches for the design of occupational case-control studies are reviewed. An accompanying paper reviews methods of analysis. We commence by drawing a distinction between cohort-based and registry-based studies. Methods for selecting cases and controls are then reviewed, including cumulative incidence and incidence density sampling, matching, sources of controls, and issues in control selection. Finally, the advantages and disadvantages of the case-control approach are summarized.     

Exploiting gene-environment independence in haplotype-based inferences for population-based case-control studies with complex sampling

The use of complex sampling in population-based case-control studies is becoming more common. Although most single nucleotide polymorphism-based association studies with complex sampling account for the design complications, many of haplotype-based genetic association studies with complex sampling tend to ignore them when estimating haplotype frequencies, regression coefficients, or both. In this article, we develop innovative one-step and two-step statistical methods that account for the design complications in haplotype-based association studies when cases and/or controls are sampled with complex sampling. Attracted by the efficiency advantage of the retrospective method, we explore the assumptions of Hardy-Weinberg equilibrium and gene-environment independence in the underlying population. Results of our simulation studies demonstrate superior performance of the proposed methods over selected existing methods under various complex sampling designs. An application of the proposed methods is illustrated using a population-based case-control study of kidney cancer.     

The experience of using random digit dialling methods in a population-based chronic pain study

OBJECTIVES: To describe response rates and call characteristics using random digit dialling (RDD) methods in a population-based prevalence study of chronic pain; to compare respondent characteristics according to telephone number listing status. METHODS: A RDD telephone study of chronic pain was conducted using computer-assisted telephone interview (CATI) methods in the Northern Sydney Area in 1998. RESULTS: Unlisted number respondents (18.5%) were younger, had a higher proportion of males, and were less likely to live in households with three or more eligible adults. There was no difference in chronic pain status between the two groups. The overall response rate was 73.4% (unlisted number group 66.3%; listed number group 75.0%). Answering machine messages boosted response rates (79.7%). 10.9% of unlisted numbers used in the study resulted in completed interviews, compared with 31.3% of the listed numbers used. CONCLUSIONS: The relatively high response rate obtained, differences in characteristics associated with listing status and reduction in sampling bias provided by using RDD methods have to be balanced against the differential response rates between listed and unlisted number groups and higher costs (including opportunity costs). IMPLICTIONS: Published data on the experience of using RDD methods can assist public health researchers in deciding whether to use these methods in telephone surveys.     

Improving the efficiency of nested case-control studies of interaction by selecting controls using counter matching on exposure

BACKGROUND: Studies of the effect of exposure to a risk factor measured in an entire cohort may be augmented by nested case-control subsets to investigate confounding or effect modification by additional factors not practically assessed on all cohort members. We compared three control-selection strategies-matching on exposure, counter matching on exposure, and random sampling-to determine which was most efficient in a situation where exposure is a known, continuous variable and high doses are rare. METHODS: We estimated the power to detect interaction using four control-to-case ratios (1:1, 2:1, 4:1, and 8:1) in a planned case-control study of the joint effect of atomic bomb radiation exposure and serum oestradiol levels on breast cancer. Radiation dose is measured in the entire cohort, but because neither serum oestradiol level nor the true degree of interaction was known, we simulated values of oestradiol and hypothetical levels of oestradiol-radiation interaction. RESULTS: Compared with random sampling, power to detect interaction was similarly higher with either matching or counter matching with two or more controls. CONCLUSIONS: Because counter matching is generally at least as efficient as random sampling, whereas matching on exposure can result in loss of efficiency and precludes estimation of exposure risk, we recommend counter matching for selecting controls in nested case-control studies of the joint effects of multiple risk factors when one is previously measured in the full cohort.     

Comparison of telephone sampling and area sampling: response rates and within-household coverage

Random digit dialing is used frequently in epidemiologic case-control studies to select population-based controls, even when both cases and controls are interviewed face-to-face. However, concerns persist about the potential biases of random digit dialing, particularly given its generally lower response rates. In an Atlanta, Georgia, case-control study of breast cancer among women aged 20-54 years, all of whom were interviewed face-to-face, two statistically independent control groups were compared: those obtained through random digit dialing (n = 652) and those obtained through area probability sampling (n = 640). The household screening rate was significantly higher for the area sample, by 5.5%. Interview response rates were comparable. The telephone sample estimated a significantly larger percentage (by approximately 7%) of households to have no age-eligible women. Both control groups, appropriately weighted, had characteristics similar to US Census demographic characteristics for Atlanta women, except that respondents in both control groups were more educated and more likely to be married. The authors conclude that households contacted through random digit dialing are somewhat less likely to participate in the household screening process, and if they are cooperative, some households may not disclose that age-eligible women reside therein. Investigators need to develop improved methods for screening and enumerating household members in random digit dialing surveys that target a specific subpopulation, such as women.     

Secondary matching: a method for selecting controls in case-control studies on environmental risk factors

BACKGROUND: The problem of control selection was considered in a population-based case-control study on pleural mesothelioma in Spain. Random sampling from the population was discarded because of potential selection bias due to low participation. Selection of hospital controls by matching them to cases by hospital seemed unsuitable for investigating environmental exposures, as the choice of hospital may be related to the place of residence; controlling for residence may avoid bias but could produce overmatching. METHODS: A three-step procedure was proposed. First, a random sample of primary controls from the population census of the province of Barcelona was obtained. Second, the hospital closest to the residence of the primary control was identified as the control hospital. Third a secondary control was chosen among patients admitted to the hospital matched to the primary control by sex, age and municipality. RESULTS: An overall participation rate of 85% was achieved. The hospital control group showed a distribution of residences similar to that of the general population, and independent of the distribution of cases. CONCLUSIONS: This procedure may be considered as an alternative for control selection when studying environmental factors or, generally, when matching cases and controls by hospital is to be avoided. Its validity was assessed according to the principles of comparability with cases regarding the study base, accuracy of information, deconfounding and efficiency.     

Explaining discrepancies in reproductive health indicators from population-based surveys and exit surveys: a case from Rwanda

OBJECTIVES: Reproductive health programmes often need exit surveys and population-based surveys for monitoring and evaluation. This study investigates why such studies produce discrepant estimates of condom use, sexual behaviour and condom brand knowledge, and discusses the implications for future use of exit surveys for programme monitoring. METHODS: Logistic regression is used to explain differences between a household survey of 1295 persons and an exit survey among a random sample of 2550 consumers at retail outlets in RWANDA: RESULTS: Discrepancies in ever use of condoms and risky sexual behaviours are due to differences in socioeconomic status of the two samples. After controls, exit surveys at most outlet types have the same results as the household survey. Only exit surveys at bars, nightclubs and hotels yield significantly different estimates. However, the above-average knowledge of Prudence Plus condoms in the exit interviews is not attributable to socioeconomic or demographic variables, most likely because respondents have seen the product at the outlets. CONCLUSIONS: Information about condom use and sexual behaviour obtained from exit surveys appears as accurate as that obtained through household surveys. Nevertheless, exit surveys must be used cautiously. Because exit surveys may include wealthier and better-educated respondents, they are not representative of the general population. The composition of exit survey samples should be validated through existing household surveys. Comparisons across survey types are generally unadvisable, unless they control for sample differences. When generalizing to the population at large is not needed (e.g. for studies aimed at identifying the characteristics and behaviour of users of particular products or services), exit surveys can provide an appropriate alternative to household surveys.     

Complementing random-digit-dial telephone surveys with other approaches to collecting sensitive data

Surveys of sensitive topics, such as the Injury Control and Risk Surveys (ICARIS) or the Behavioral Risk Factors Surveillance System (BRFSS), are often conducted by telephone using random-digit-dial (RDD) sampling methods. Although this method of data collection is relatively quick and inexpensive, it suffers from growing coverage problems and falling response rates. In this paper, several alternative methods of data collection are reviewed, including audio computer-assisted interviews as part of personal visit surveys, mail surveys, web surveys, and interactive voice response surveys. Their strengths and weaknesses are presented regarding coverage, nonresponse, and measurement issues, and compared with RDD telephone surveys. The feasibility of several mixed mode designs is discussed; none of them stands out as clearly the right choice for surveys on sensitive issues, which implies increased need for methodologic research.     

The future of random-digit-dial surveys for injury prevention and violence research

A central issue facing injury prevention research today is how to collect self-reported data on injury and violence from a geographically dispersed public, quickly, cost effectively, and with a reasonable degree of confidence in the quality of the results. Questions about eroding frame coverage, declining participation rates, and increasing potential for bias have raised doubts about the long-term viability of random-digit-dial (RDD) telephone surveys for injury prevention research. So where does the future lie? The four articles in this volume, as well as other research, point down two paths: (1) continued reliance on RDD, or (2) adoption of alternative survey designs. Continued use of RDD methodology will require additional research in the areas of response rate improvement, techniques for enhancing post-survey adjustments, and cost-effective approaches to nonresponse bias analysis. Moving away from a strict reliance on RDD methodology, injury prevention research could adopt mixed-mode approaches (such as combining telephone, mail, and web-based surveys) or make use of address-based sampling frames as a method for reaching sample members currently missed by most RDD approaches. Either way, the future of collecting self-reports of injury and injury prevention data will be more complex and require considerable resources.     

Exposure opportunity in case-control studies

Efficient control of confounding is well recognized as a legitimate motive for restriction, matching, and conditional analysis in case-control studies. Some investigators, however, have advocated use of the same techniques for another purpose: to achieve a balance of "opportunity" or "potential" for past exposure between cases and controls. This paper shows that disparities of exposure opportunity between cases and controls exert no bias on estimates of the incidence rate ratio. The precision of rate ratio estimates is lessened when case-control studies are reduced in size by the exclusion of people with no opportunity for exposure. Matching and conditional analysis with respect to indicators of exposure opportunity also reduce precision without enhancing validity. Such indicators, which are correlates of exposure but not determinants of disease, are not confounders; therefore, they do not need to be controlled in the design or analysis of case-control studies.     

匹配在观察性研究中的作用——有向无环图视角

匹配是观察性研究中选择研究对象的一种常用方法,具有控制混杂因素、提高统计效率等作用,但其控制混杂因素的作用在不同观察性研究中并不一致,匹配在队列研究中能够消除匹配变量的混杂偏倚,但在病例对照研究中匹配本身并不能消除混杂偏倚。在匹配性病例对照研究选择匹配变量时,研究者可能并不能准确判断该变量是否为混杂变量,若误将真实情况...     

Applying sample survey methods to clinical trials data

This paper outlines the utility of statistical methods for sample surveys in analysing clinical trials data. Sample survey statisticians face a variety of complex data analysis issues deriving from the use of multi-stage probability sampling from finite populations. One such issue is that of clustering of observations at the various stages of sampling. Survey data analysis approaches developed to accommodate clustering in the sample design have more general application to clinical studies in which repeated measures structures are encountered. Situations where these methods are of interest include multi-visit studies where responses are observed at two or more time points for each patient, multi-period cross-over studies, and epidemiological studies for repeated occurrences of adverse events or illnesses. We describe statistical procedures for fitting multiple regression models to sample survey data that are more effective for repeated measures studies with complicated data structures than the more traditional approaches of multivariate repeated measures analysis. In this setting, one can specify a primary sampling unit within which repeated measures have intraclass correlation. This intraclass correlation is taken into account by sample survey regression methods through robust estimates of the standard errors of the regression coefficients. Regression estimates are obtained from model fitting estimation equations which ignore the correlation structure of the data (that is, computing procedures which assume that all observational units are independent or are from simple random samples). The analytic approach is straightforward to apply with logistic models for dichotomous data, proportional odds models for ordinal data, and linear models for continuously scaled data, and results are interpretable in terms of population average parameters. Through the features summarized here, the sample survey regression methods have many similarities to the broader family of methods based on generalized estimating equations (GEE). Sample survey methods for the analysis of time-to-event data have more recently been developed and implemented in the context of finite probability sampling. Given the importance of survival endpoints in late phase studies for drug development, these methods have clear utility in the area of clinical trials data analysis. A brief overview of methods for sample survey data analysis is first provided, followed by motivation for applying these methods to clinical trials data. Examples drawn from three clinical studies are provided to illustrate survey methods for logistic regression, proportional odds regression and proportional hazards regression. Potential problems with the proposed methods and ways of addressing them are discussed.