Sampling Patients within Physician Practices and Health Plans: Multistage Cluster Samples in Health Services Research

Objective. To better inform study design decisions when sampling patients within health plans and physician practices with multiple analysis goals.
Study Setting. Chronic eye care patients within six health plans across the United States.
Study Design. We developed a simulation-based approach for designing multistage samples. We created a range of candidate designs, evaluated them with respect to multiple sampling goals, investigated their tradeoffs, and identified the design that is the best compromise among all goals. This approach recognizes that most data collection efforts have multiple competing goals.
Data Collection. We constructed a sample frame from all diabetic patients in six health plans with evidence of chronic eye disease (glaucoma and retinopathy).
Principal Findings. Simulations of different study designs can uncover efficiency gains as well as inform potential tradeoffs among study goals. Simulations enable us to quantify these efficiency gains and to draw tradeoff curves.
Conclusions. When designing a complex multistage sample it is desirable to explore the tradeoffs between competing sampling goals via simulation. Simulations enable us to investigate a larger number of candidate designs and are therefore likely to identify more efficient designs.     

Response‐adaptive decision‐theoretic trial design: operating characteristics and ethics

Adaptive randomization is used in clinical trials to increase statistical efficiency. In addition, some clinicians and researchers believe that using adaptive randomization leads necessarily to more ethical treatment of subjects in a trial. We develop Bayesian, decision‐theoretic, clinical trial designs with response‐adaptive randomization and a primary goal of estimating treatment effect and then contrast these designs with designs that also include in their loss function a cost for poor subject outcome. When the loss function did not incorporate a cost for poor subject outcome, the gains in efficiency from response‐adaptive randomization were accompanied by ethically concerning subject allocations. Conversely, including a cost for poor subject outcome demonstrated a more acceptable balance between the competing needs in the trial. A subsequent, parallel set of trials designed to control explicitly types I and II error rates showed that much of the improvement achieved through modification of the loss function was essentially negated. Therefore, gains in efficiency from the use of a decision‐theoretic, response‐adaptive design using adaptive randomization may only be assumed to apply to those goals that are explicitly included in the loss function. Trial goals, including ethical ones, which do not appear in the loss function, are ignored and may even be compromised; it is thus inappropriate to assume that all adaptive trials are necessarily more ethical. Controlling types I and II error rates largely negates the benefit of including competing needs in favor of the goal of parameter estimation. Copyright © 2013 John Wiley & Sons, Ltd.     

Exploring the benefits of adaptive sequential designs in time-to-event endpoint settings

Sequential analysis is frequently employed to address ethical and financial issues in clinical trials. Sequential analysis may be performed using standard group sequential designs, or, more recently, with adaptive designs that use estimates of treatment effect to modify the maximal statistical information to be collected. In the general setting in which statistical information and clinical trial costs are functions of the number of subjects used, it has yet to be established whether there is any major efficiency advantage to adaptive designs over traditional group sequential designs. In survival analysis, however, statistical information (and hence efficiency) is most closely related to the observed number of events, while trial costs still depend on the number of patients accrued. As the number of subjects may dominate the cost of a trial, an adaptive design that specifies a reduced maximal possible sample size when an extreme treatment effect has been observed may allow early termination of accrual and therefore a more cost-efficient trial. We investigate and compare the tradeoffs between efficiency (as measured by average number of observed events required), power, and cost (a function of the number of subjects accrued and length of observation) for standard group sequential methods and an adaptive design that allows for early termination of accrual. We find that when certain trial design parameters are constrained, an adaptive approach to terminating subject accrual may improve upon the cost efficiency of a group sequential clinical trial investigating time-to-event endpoints. However, when the spectrum of group sequential designs considered is broadened, the advantage of the adaptive designs is less clear.     

Choosing profile double‐sampling designs for survival estimation with application to President's Emergency Plan for AIDS Relief evaluation

Most studies that follow subjects over time are challenged by having some subjects who dropout. Double sampling is a design that selects and devotes resources to intensively pursue and find a subset of these dropouts, then uses data obtained from these to adjust naïve estimates, which are potentially biased by the dropout. Existing methods to estimate survival from double sampling assume a random sample. In limited‐resource settings, however, generating accurate estimates using a minimum of resources is important. We propose using double‐sampling designs that oversample certain profiles of dropouts as more efficient alternatives to random designs. First, we develop a framework to estimate the survival function under these profile double‐sampling designs. We then derive the precision of these designs as a function of the rule for selecting different profiles, in order to identify more efficient designs. We illustrate using data from the United States President's Emergency Plan for AIDS Relief‐funded HIV care and treatment program in western Kenya. Our results show why and how more efficient designs should oversample patients with shorter dropout times. Further, our work suggests generalizable practice for more efficient double‐sampling designs, which can help maximize efficiency in resource‐limited settings. Copyright © 2014 John Wiley & Sons, Ltd.     

Immunization coverage survey: empirical assessment of the cluster sampling method proposed by the World Health Organization]

OBJECTIVE: To assess sample representativeness and the precision of estimates of immunization coverage obtained with the 30 by 7 cluster sampling method proposed by the World Health Organization, by applying the method to determine immunization coverage in two municipalities (Diadema and S?o Caetano do Sul) in the state of S?o Paulo, Brazil, in 2000. METHOD: The representativeness of the samples was determined by comparing the census sectors picked by lot for the surveyed sectors and for the nonsurveyed sectors in both municipalities, in terms of socioeconomic and demographic characteristics (age distribution of the population, schooling, proportion of households headed by a women, monthly income of household head, and sanitary conditions of the home (piped-in water, connected to the sewer system)). The precision of the coverage estimates for the vaccines in the basic immunization schedule-BCG; diphtheria, pertussis, and tetanus (DPT); poliomyelitis; hepatitis B; measles; and measles, mumps, and rubella (MMR)-was determined by calculating the design effect and the width of the confidence intervals. Precision was considered to be satisfactory if the design effect was below 2.0 and the confidence interval width was below 10%. RESULTS: In both municipalities the comparison between the surveyed and nonsurveyed sectors showed a similar distribution in terms of socioeconomic and demographic variables. Concerning the precision of the estimates, the design effect was below 2.0 for all the vaccines, both in S?o Caetano do Sul and Diadema. In Diadema, the confidence interval width was below 10% for all the vaccines, except for MMR (10.1%). In S?o Caetano do Sul, only 89% of the expected sample were included, so the width of the confidence interval was slightly above 10% for the poliomyelitis vaccine (10.3%), the hepatitis B vaccine (11.8%), the mumps vaccine (10.4%), the MMR (12.9%), and the complete schedule (11.2%). CONCLUSION: The cluster sampling method proposed by the World Health Organization produces representative data as long as the methodological procedures for selecting the sample are rigorously followed in the field.     

Semiparametric odds ratio model for case–control and matched case–control designs

We propose a semiparametric odds ratio model that extends Umbach and Weinberg's approach to exploiting gene–environment association model for efficiency gains in case–control designs to both discrete and continuous data. We directly model the gene–environment association in the control population to avoid estimating the intercept in the disease risk model, which is inherently difficult because of the scarcity of information on the parameter with the sampling designs. We propose a novel permutation‐based approach to eliminate the high‐dimensional nuisance parameters in the matched case–control design. The proposed approach reduces to the conditional logistic regression when the model for the gene–environment association is unrestricted. Simulation studies demonstrate good performance of the proposed approach. We apply the proposed approach to a study of gene–environment interaction on coronary artery disease. Copyright © 2013 John Wiley & Sons, Ltd.     

Exposure enriched outcome dependent designs for longitudinal studies of gene–environment interaction

Joint effects of genetic and environmental factors have been increasingly recognized in the development of many complex human diseases. Despite the popularity of case‐control and case‐only designs, longitudinal cohort studies that can capture time‐varying outcome and exposure information have long been recommended for gene–environment (G × E) interactions. To date, literature on sampling designs for longitudinal studies of G × E interaction is quite limited. We therefore consider designs that can prioritize a subsample of the existing cohort for retrospective genotyping on the basis of currently available outcome, exposure, and covariate data. In this work, we propose stratified sampling based on summaries of individual exposures and outcome trajectories and develop a full conditional likelihood approach for estimation that adjusts for the biased sample. We compare the performance of our proposed design and analysis with combinations of different sampling designs and estimation approaches via simulation. We observe that the full conditional likelihood provides improved estimates for the G × E interaction and joint exposure effects over uncorrected complete‐case analysis, and the exposure enriched outcome trajectory dependent design outperforms other designs in terms of estimation efficiency and power for detection of the G × E interaction. We also illustrate our design and analysis using data from the Normative Aging Study, an ongoing longitudinal cohort study initiated by the Veterans Administration in 1963. Copyright © 2017 John Wiley & Sons, Ltd.     

Parametric empirical Bayes estimates of disease prevalence using stratified samples from community populations

Studies of chronic diseases in a community setting often employ stratified sample designs to enable the study to attain multiple research goals at a reasonable cost. One important goal is estimation of disease prevalence in the whole community and in important subgroups. Some adjustment for the sample design is necessary; if the design has many strata with very disparate sampling fractions, simply upweighting observed stratum prevalences may lead to unstable estimators. We propose a parametric empirical Bayes estimator in the spirit of the work of Efron and Morris, and we compare it to the direct upweighted estimator and a regression-smoothed estimator. Simulation studies in realistic settings suggest that the new estimator performs best, giving estimates with low bias and good precision under a variety of models.     

The impact of survey design modifications on health insurance coverage estimates in a National Longitudinal Health Care Survey

National health insurance coverage estimates for the overall population and specific population subgroups are critical to policymakers and others concerned with access to medical care and the cost and sources of payment for that care. The Medical Expenditure Panel Survey (MEPS) is one of the core health care surveys in the United States that serves as a primary source for these essential national health insurance coverage estimates. The survey is designed to provide annual national estimates of the health care use, medical expenditures, sources of payment and insurance coverage for the U.S. civilian non-institutionalized population. In 2007, the survey experienced two dominant survey design modifications: (1) a new sample design attributable to the sample redesign of the National Health Interview Survey, and (2) an upgrade to the CAPI platform for the survey instrument, moving from a DOS to a Windows based environment. This study examines the impact of these survey design modifications on the national estimates of insurance coverage. The overlapping panel design of the MEPS survey and its longitudinal features are particularly well suited to assess the impact of survey redesign modifications on estimates. Since two independent nationally representative samples are pooled to produce calendar year estimates, one has the capacity to compare estimates based on the “original survey design” in contrast to those derived from the “survey redesign.” This paper examines the correlates of nonresponse incorporated in the estimation techniques and adjustment methods employed in the survey, and the measures utilized for post-stratification overall and by panel. Particular attention is given to assessing the level of convergence in coverage estimates based on the alternative designs as well as the alignment of model based analyses that discern which factors are associated with health insurance classifications. The paper concludes with a discussion of strategies under consideration that may yield additional improvements in the accuracy for these critical policy relevant survey estimates.     

Selection within households in health surveys

OBJECTIVE

To compare the efficiency and accuracy of sampling designs including and excluding the sampling of individuals within sampled households in health surveys.

METHODS

From a population survey conducted in Baixada Santista Metropolitan Area, SP, Southeastern Brazil, lowlands between 2006 and 2007, 1,000 samples were drawn for each design and estimates for people aged 18 to 59 and 18 and over were calculated for each sample. In the first design, 40 census tracts, 12 households per sector, and one person per household were sampled. In the second, no sampling within the household was performed and 40 census sectors and 6 households for the 18 to 59-year old group and 5 or 6 for the 18 and over age group or more were sampled. Precision and bias of proportion estimates for 11 indicators were assessed in the two final sets of the 1000 selected samples with the two types of design. They were compared by means of relative measurements: coefficient of variation, bias/mean ratio, bias/standard error ratio, and relative mean square error. Comparison of costs contrasted basic cost per person, household cost, number of people, and households.

RESULTS

Bias was found to be negligible for both designs. A lower precision was found in the design including individuals sampling within households, and the costs were higher.

CONCLUSIONS

The design excluding individual sampling achieved higher levels of efficiency and accuracy and, accordingly, should be first choice for investigators. Sampling of household dwellers should be adopted when there are reasons related to the study subject that may lead to bias in individual responses if multiple dwellers answer the proposed questionnaire.     

Optimal auxiliary‐covariate‐based two‐phase sampling design for semiparametric efficient estimation of a mean or mean difference,with application to clinical trials

To address the objective in a clinical trial to estimate the mean or mean difference of an expensive endpoint Y, one approach employs a two‐phase sampling design, wherein inexpensive auxiliary variables W predictive of Y are measured in everyone, Y is measured in a random sample, and the semiparametric efficient estimator is applied. This approach is made efficient by specifying the phase two selection probabilities as optimal functions of the auxiliary variables and measurement costs. While this approach is familiar to survey samplers, it apparently has seldom been used in clinical trials, and several novel results practicable for clinical trials are developed. We perform simulations to identify settings where the optimal approach significantly improves efficiency compared to approaches in current practice. We provide proofs and R code. The optimality results are developed to design an HIV vaccine trial, with objective to compare the mean ‘importance‐weighted’ breadth (Y) of the T‐cell response between randomized vaccine groups. The trial collects an auxiliary response (W) highly predictive of Y and measures Y in the optimal subset. We show that the optimal design‐estimation approach can confer anywhere between absent and large efficiency gain (up to 24 % in the examples) compared to the approach with the same efficient estimator but simple random sampling, where greater variability in the cost‐standardized conditional variance of Y given W yields greater efficiency gains. Accurate estimation of E[Y | W] is important for realizing the efficiency gain, which is aided by an ample phase two sample and by using a robust fitting method. Copyright © 2013 John Wiley & Sons, Ltd.     

Bayesian adaptive N-of-1 trials for estimating population and individual treatment effects

This article proposes a novel adaptive design algorithm that can be used to find optimal treatment allocations in N-of-1 clinical trials. This new methodology uses two Laplace approximations to provide a computationally efficient estimate of population and individual random effects within a repeated measures, adaptive design framework. Given the efficiency of this approach, it is also adopted for treatment selection to target the collection of data for the precise estimation of treatment effects. To evaluate this approach, we consider both a simulated and motivating N-of-1 clinical trial from the literature. For each trial, our methods were compared with the multiarmed bandit approach and a randomized N-of-1 trial design in terms of identifying the best treatment for each patient and the information gained about the model parameters. The results show that our new approach selects designs that are highly efficient in achieving each of these objectives. As such, we propose our Laplace-based algorithm as an efficient approach for designing adaptive N-of-1 trials.     

Modeling and presentation of vaccination coverage estimates using data from household surveys

It is becoming increasingly popular to produce high-resolution maps of vaccination coverage by fitting Bayesian geostatistical models to data from household surveys. Usually, the surveys adopt a stratified cluster sampling design. We discuss a number of crucial choices with respect to two key aspects of the map production process: the acknowledgement of the survey design in modeling, and the appropriate presentation of estimates and their uncertainties. Specifically, we consider the importance of accounting for urban/rural stratification and cluster-level non-spatial excess variation in survey outcomes, when fitting geostatistical models. We also discuss the trade-off between the geographical scale and precision of model-based estimates, and demonstrate visualization methods for mapping and ranking that emphasize the probabilistic interpretation of results. A novel approach to coverage map presentation is proposed to allow comparison and control of the overall map uncertainty. We use measles vaccination coverage in Nigeria as a motivating example and illustrate the different issues using data from the 2018 Nigeria Demographic and Health Survey.     

Profiling the mobile‐only population in Australia: insights from the Australian National Health Survey

Background: The Australian population that relies on mobile phones exclusively has increased from 5% in 2005 to 29% in 2014. Failing to include this mobile‐only population leads to a potential bias in estimates from landline‐based telephone surveys. This paper considers the impacts on selected health prevalence estimates with and without the mobile‐only population. Methods: Using data from the Australian Health Survey – which, for the first time, included a question on telephone status – we examined demographic, geographic and health differences between the landline‐accessible and mobile‐only population. These groups were also compared to the full population, controlling for the sampling design and differential non‐response patterns in the observed sample through weighting and benchmarking. Results: The landline‐accessible population differs from the mobile‐only population for selected health measures resulting in biased prevalence estimates for smoking, alcohol risk and private health insurance coverage in the full population. The differences remain even after adjusting for age and gender. Conclusions: Using landline telephones only for conducting population health surveys will have an impact on prevalence rate estimates of health risk factors due to the differing profiles of the mobile‐only population from the landline‐accessible population.     

Cohort versus cross-sectional design in large field trials: Precision,sample size,and a unifying model

In planning large longitudinal field trials, one is often faced with a choice between a cohort design and a cross-sectional design, with attendant issues of precision, sample size, and bias. To provide a practical method for assessing these trade-offs quantitatively, we present a unifying statistical model that embraces both designs as special cases. The model takes account of continuous and discrete endpoints, site differences, and random cluster and subject effects of both a time-invariant and a time-varying nature. We provide a comprehensive design equation, relating sample size to precision for cohort and cross-sectional designs, and show that the follow-up cost and selection bias attending a cohort design may outweigh any theoretical advantage in precision. We provide formulae for the minimum number of clusters and subjects. We relate this model to the recently published prevalence model for COMMIT, a multi-site trial of smoking cessation programmes. Finally, we tabulate parameter estimates for some physiological endpoints from recent community-based heart-disease prevention trials, work an example, and discuss the need for compiling such estimates as a basis for informed design of future field trials.     

Weighing gains and losses in criterion validity studies with test-based enrollment

Criterion validity studies sometimes use designs with test-based enrollment schemes. In such incomplete studies, a Reference Instrument (RI) is applied to unequal sampling fractions of subjects previously identified as positives or negatives by a new Test Instrument (TI+ and TI-). Focusing on sensitivity (Se) and specificity (Sp), this article addresses some issues concerning the precision of estimates, study costs, as well as the acceptability/convenience to subjects. For that purpose, examples are provided whereby three indicators-statistical efficiency differential (deltaS), cost differential (deltaC), and (in)convenience differential (deltaI)-are contrasted and discussed. Although a clear, fast-and-ready answer as to what constitutes an optimal study cannot be given, the article offers a rationale for weighing gains and losses. Among several scenarios, it is shown that an appropriately chosen incomplete study design may be as statistically efficient as one with a complete sampling scheme, yet is able to offer a ca. 15% cost reduction and about 20% fewer individuals needing to endure an invasive or logistically cumbersome RI. A special emphasis on the planning stages of an investigation is called for, precisely when the level of statistical precision the researcher is willing to accept can be weighed against the available budget and the degree of stress put on the subject that ought to be avoided.     

Stratified case sampling and the use of family controls

We compare the asymptotic relative efficiency (ARE) of different study designs for estimating gene and gene-environment interaction effects using matched case-control data. In the sampling schemes considered, cases are selected differentially based on their family history of disease. Controls are selected either from unrelated subjects or from among the case's unaffected siblings and cousins. Parameters are estimated using weighted conditional logistic regression, where the likelihood contributions for each subject are weighted by the fraction of cases sampled sharing the same family history. Results showed that compared to random sampling, over-sampling cases with a positive family history increased the efficiency for estimating the main effect of a gene for sib-control designs (103-254% ARE) and decreased efficiency for cousin-control and population-control designs (68-94% ARE and 67-84% ARE, respectively). Population controls and random sampling of cases were most efficient for a recessive gene or a dominant gene with an relative risk less than 9. For estimating gene-environment interactions, over-sampling positive-family-history cases again led to increased efficiency using sib controls (111-180% ARE) and decreased efficiency using population controls (68-87% ARE). Using case-cousin pairs, the results differed based on the genetic model and the size of the interaction effect; biased sampling was only slightly more efficient than random sampling for large interaction effects under a dominant gene model (relative risk ratio = 8, 106% ARE). Overall, the most efficient study design for studying gene-environment interaction was the case-sib-control design with over-sampling of positive-family-history-cases.     

Asymptotic bias and efficiency in case-control studies of candidate genes and gene-environment interactions: basic family designs

Case-control designs that use population controls are compared with those that use controls selected from their relatives (i.e., siblings, cousins, or "pseudosibs" based on parental alleles) for estimating the effect of candidate genes and gene-environment interactions. The authors first evaluate the asymptotic bias in relative risk estimates resulting from using population controls when there is confounding due to population stratification. Using siblings or pseudosibs as controls completely addresses this issue, whereas cousins provide only partial protection from population stratification. Next, they show that the conventional conditional likelihood for matched case-control studies can give asymptotically biased effect estimates when applied to the pseudosib approach; the asymptotic bias is toward the null and disappears with disease rarity. They show how to reparameterize the pseudosib likelihood so this approach gives consistent effect estimates. They then show that the designs using population or pseudosib controls are generally the most efficient for estimating the main effect of a candidate gene, followed in efficiency by the design using cousins. Finally, they show that the design using sibling controls can be quite efficient when studying gene-environment interactions. In addition to asymptotic bias and efficiency issues, family-based designs might benefit from a higher motivation to participate among cases' relatives, but these designs have the disadvantage that many potential cases will be excluded from study by having no available controls.