BinomiRare: A robust test of the association of a rare variant with a disease for pooled analysis and meta‐analysis,with application to the HCHS/SOL

Most regression‐based tests of the association between a low‐count variant and a binary outcome do not protect type 1 error, especially when tests are rejected based on a very low significance threshold. Noted exception is the Firth test. However, it was recently shown that in meta‐analyzing multiple studies all asymptotic, regression‐based tests, including the Firth, may not control type 1 error in some settings, and the Firth test may suffer a substantial loss of power. The problem is exacerbated when the case‐control proportions differ between studies. I propose the BinomiRare exact test that circumvents the calibration problems of regression‐based estimators. It quantifies the strength of association between the variant and the disease outcome based on the departure of the number of diseased individuals carrying the variant from the expected distribution of disease probability, under the null hypothesis of no association between the disease outcome and the rare variant. I provide a meta‐analytic strategy to combine tests across multiple cohorts, which requires that each cohort provides the disease probabilities of all carriers of the variant in question, and the number of diseased individuals among the carriers. I show that BinomiRare controls type 1 error in meta‐analysis even when the case‐control proportions differ between the studies, and does not lose power compared to pooled analysis. I demonstrate the test in studying the association of rare variants with asthma in the Hispanic Community Health Study/Study of Latinos.     

An approximate approach to sample size determination in bioequivalence testing with multiple pharmacokinetic responses

The approval of generic drugs requires the evidence of average bioequivalence (ABE) on both the area under the concentration–time curve and the peak concentration C_max. The bioequivalence (BE) hypothesis can be decomposed into the non‐inferiority (NI) and non‐superiority (NS) hypothesis. Most of regulatory agencies employ the two one‐sided tests (TOST) procedure to test ABE between two formulations. As it is based on the intersection–union principle, the TOST procedure is conservative in terms of the type I error rate. However, the type II error rate is the sum of the type II error rates with respect to each null hypothesis of NI and NS hypotheses. When the difference in population means between two treatments is not 0, no close‐form solution for the sample size for the BE hypothesis is available. Current methods provide the sample sizes with either insufficient power or unnecessarily excessive power. We suggest an approximate method for sample size determination, which can also provide the type II rate for each of NI and NS hypotheses. In addition, the proposed method is flexible to allow extension from one pharmacokinetic (PK) response to determination of the sample size required for multiple PK responses. We report the results of a numerical study. An R code is provided to calculate the sample size for BE testing based on the proposed methods. Copyright © 2014 John Wiley & Sons, Ltd.     

P-value based analysis for shared controls design in genome-wide association studies

An appealing genome-wide association study design compares one large control group against several disease samples. A pioneering study by the Wellcome Trust Case Control Consortium that employed such a design has identified multiple susceptibility regions, many of which have been independently replicated. While reusing a control sample provides effective utilization of data, it also creates correlation between association statistics across diseases. An observation of a large association statistic for one of the diseases may greatly increase chances of observing a spuriously large association for a different disease. Accounting for the correlation is also particularly important when screening for SNPs that might be involved in a set of diseases with overlapping etiology. We describe methods that correct association statistics for dependency due to shared controls, and we describe ways to obtain a measure of overall evidence and to combine association signals across multiple diseases. The methods we describe require no access to individual subject data, instead, they efficiently utilize information contained in P-values for association reported for individual diseases. P-value based combined tests for association are flexible and essentially as powerful as the approach based on aggregating the individual subject data.     

Analysis goals,error‐cost sensitivity,and analysis hacking: Essential considerations in hypothesis testing and multiple comparisons

The “replication crisis” has been attributed to perverse incentives that lead to selective reporting and misinterpretations of P‐values and confidence intervals. A crude fix offered for this problem is to lower testing cut‐offs (α levels), either directly or in the form of null‐biased multiple comparisons procedures such as naïve Bonferroni adjustments. Methodologists and statisticians have expressed positions that range from condemning all such procedures to demanding their application in almost all analyses. Navigating between these unjustifiable extremes requires defining analysis goals precisely enough to separate inappropriate from appropriate adjustments. To meet this need, I here review issues arising in single‐parameter inference (such as error costs and loss functions) that are often skipped in basic statistics, yet are crucial to understanding controversies in testing and multiple comparisons. I also review considerations that should be made when examining arguments for and against modifications of decision cut‐offs and adjustments for multiple comparisons. The goal is to provide researchers a better understanding of what is assumed by each side and to enable recognition of hidden assumptions. Basic issues of goal specification and error costs are illustrated with simple fixed cut‐off hypothesis testing scenarios. These illustrations show how adjustment choices are extremely sensitive to implicit decision costs, making it inevitable that different stakeholders will vehemently disagree about what is necessary or appropriate. Because decisions cannot be justified without explicit costs, resolution of inference controversies is impossible without recognising this sensitivity. Pre‐analysis statements of funding, scientific goals, and analysis plans can help counter demands for inappropriate adjustments, and can provide guidance as to what adjustments are advisable. Hierarchical (multilevel) regression methods (including Bayesian, semi‐Bayes, and empirical‐Bayes methods) provide preferable alternatives to conventional adjustments, insofar as they facilitate use of background information in the analysis model, and thus can provide better‐informed estimates on which to base inferences and decisions.     

Improving power in genome-wide association studies: weights tip the scale

The potential of genome-wide association analysis can only be realized when they have power to detect signals despite the detrimental effect of multiple testing on power. We develop a weighted multiple testing procedure that facilitates the input of prior information in the form of groupings of tests. For each group a weight is estimated from the observed test statistics within the group. Differentially weighting groups improves the power to detect signals in likely groupings. The advantage of the grouped-weighting concept, over fixed weights based on prior information, is that it often leads to an increase in power even if many of the groupings are not correlated with the signal. Being data dependent, the procedure is remarkably robust to poor choices in groupings. Power is typically improved if one (or more) of the groups clusters multiple tests with signals, yet little power is lost when the groupings are totally random. If there is no apparent signal in a group, relative to a group that appears to have several tests with signals, the former group will be down-weighted relative to the latter. If no groups show apparent signals, then the weights will be approximately equal. The only restriction on the procedure is that the number of groups be small, relative to the total number of tests performed.     

A method for testing a prespecified subgroup in clinical trials

In clinical trials, investigators are often interested in the effect of a given study treatment on a subgroup of patients with certain clinical or biological attributes in addition to its effect on the overall study population. Such a subgroup analysis would become even more important to the study sponsor if an efficacy claim can be made for the subgroup when the test for the overall study population fails at a prespecified alpha level. In practice, such a claim is often dependent on prespecification of the subgroup and certain implicit or explicit requirements placed on the study results due to ethical or regulatory concerns. By carefully considering these requirements, we propose a general statistical methodology for testing both the overall and subgroup hypotheses, which has optimal power and strongly controls the familywise Type I error rate.     

Permutation-based adjustments for the significance of partial regression coefficients in microarray data analysis

The aim of this paper is to generalize permutation methods for multiple testing adjustment of significant partial regression coefficients in a linear regression model used for microarray data. Using a permutation method outlined by Anderson and Legendre [1999] and the permutation P-value adjustment from Simon et al. [2004], the significance of disease related gene expression will be determined and adjusted after accounting for the effects of covariates, which are not restricted to be categorical. We apply these methods to a microarray dataset containing confounders and illustrate the comparisons between the permutation-based adjustments and the normal theory adjustments. The application of a linear model is emphasized for data containing confounders and the permutation-based approaches are shown to be better suited for microarray data.     

Symmetric graphs for equally weighted tests,with application to the Hochberg procedure

The graphical approach to multiple testing provides a convenient tool for designing, visualizing, and performing multiplicity adjustments in confirmatory clinical trials while controlling the familywise error rate. It assigns a set of weights to each intersection null hypothesis within the closed test framework. These weights form the basis for intersection tests using weighted individual p-values, such as the weighted Bonferroni test. In this paper, we extend the graphical approach to intersection tests that assume equal weights for the elementary null hypotheses associated with any intersection hypothesis, including the Hochberg procedure as well as omnibus tests such as Fisher's combination, O'Brien's, and F tests. More specifically, we introduce symmetric graphs that generate sets of equal weights so that the aforementioned tests can be applied with the graphical approach. In addition, we visualize the Hochberg and the truncated Hochberg procedures in serial and parallel gatekeeping settings using symmetric component graphs. We illustrate the method with two clinical trial examples.     

Stratified false discovery control for large-scale hypothesis testing with application to genome-wide association studies

The multiplicity problem has become increasingly important in genetic studies as the capacity for high-throughput genotyping has increased. The control of False Discovery Rate (FDR) (Benjamini and Hochberg. [1995] J. R. Stat. Soc. Ser. B 57:289-300) has been adopted to address the problems of false positive control and low power inherent in high-volume genome-wide linkage and association studies. In many genetic studies, there is often a natural stratification of the m hypotheses to be tested. Given the FDR framework and the presence of such stratification, we investigate the performance of a stratified false discovery control approach (i.e. control or estimate FDR separately for each stratum) and compare it to the aggregated method (i.e. consider all hypotheses in a single stratum). Under the fixed rejection region framework (i.e. reject all hypotheses with unadjusted p-values less than a pre-specified level and then estimate FDR), we demonstrate that the aggregated FDR is a weighted average of the stratum-specific FDRs. Under the fixed FDR framework (i.e. reject as many hypotheses as possible and meanwhile control FDR at a pre-specified level), we specify a condition necessary for the expected total number of true positives under the stratified FDR method to be equal to or greater than that obtained from the aggregated FDR method. Application to a recent Genome-Wide Association (GWA) study by Maraganore et al. ([2005] Am. J. Hum. Genet. 77:685-693) illustrates the potential advantages of control or estimation of FDR by stratum. Our analyses also show that controlling FDR at a low rate, e.g. 5% or 10%, may not be feasible for some GWA studies.     

A statistical model for measurement error that incorporates variation over time in the target measure,with application to nutritional epidemiology

Most statistical methods that adjust analyses for measurement error assume that the target exposure T is a fixed quantity for each individual. However, in many applications, the value of T for an individual varies with time. We develop a model that accounts for such variation, describing the model within the framework of a meta‐analysis of validation studies of dietary self‐report instruments, where the reference instruments are biomarkers. We demonstrate that in this application, the estimates of the attenuation factor and correlation with true intake, key parameters quantifying the accuracy of the self‐report instrument, are sometimes substantially modified under the time‐varying exposure model compared with estimates obtained under a traditional fixed‐exposure model. We conclude that accounting for the time element in measurement error problems is potentially important. Copyright © 2015 John Wiley & Sons, Ltd.     

A robust and powerful two‐step testing procedure for local ancestry adjusted allelic association analysis in admixed populations

Genetic association studies in admixed populations allow us to gain deeper understanding of the genetic architecture of human diseases and traits. However, population stratification, complicated linkage disequilibrium (LD) patterns, and the complex interplay of allelic and ancestry effects on phenotypic traits pose challenges in such analyses. These issues may lead to detecting spurious associations and/or result in reduced statistical power. Fortunately, if handled appropriately, these same challenges provide unique opportunities for gene mapping. To address these challenges and to take these opportunities, we propose a robust and powerful two‐step testing procedure Local Ancestry Adjusted Allelic (LAAA) association. In the first step, LAAA robustly captures associations due to allelic effect, ancestry effect, and interaction effect, allowing detection of effect heterogeneity across ancestral populations. In the second step, LAAA identifies the source of association, namely allelic, ancestry, or the combination. By jointly modeling allele, local ancestry, and ancestry‐specific allelic effects, LAAA is highly powerful in capturing the presence of interaction between ancestry and allele effect. We evaluated the validity and statistical power of LAAA through simulations over a broad spectrum of scenarios. We further illustrated its usefulness by application to the Candidate Gene Association Resource (CARe) African American participants for association with hemoglobin levels. We were able to replicate independent groups’ previously identified loci that would have been missed in CARe without joint testing. Moreover, the loci, for which LAAA detected potential effect heterogeneity, were replicated among African Americans from the Women's Health Initiative study. LAAA is freely available at https://yunliweb.its.unc.edu/LAAA .     

Adapting the logical basis of tests for Hardy‐Weinberg Equilibrium to the real needs of association studies in human and medical genetics

The standard procedure to assess genetic equilibrium is a χ² test of goodness‐of‐fit. As is the case with any statistical procedure of that type, the null hypothesis is that the distribution underlying the data is in agreement with the model. Thus, a significant result indicates incompatibility of the observed data with the model, which is clearly at variance with the aim in the majority of applications: to exclude the existence of gross violations of the equilibrium condition. In current practice, we try to avoid this basic logical difficulty by increasing the significance bound to the P‐value (e.g. from 5 to 10%) and inferring compatibility of the data with Hardy Weinberg Equilibrium (HWE) from an insignificant result. Unfortunately, such direct inversion of a statistical testing procedure fails to produce a valid test of the hypothesis of interest, namely, that the data are in sufficiently good agreement with the model under which the P‐value is calculated. We present a logically unflawed solution to the problem of establishing (approximate) compatibility of an observed genotype distribution with HWE. The test is available in one‐ and two‐sided versions. For both versions, we provide tools for exact power calculation. We demonstrate the merits of the new approach through comparison with the traditional χ² goodness‐of‐fit test in 2×60 genotype distributions from 43 published genetic studies of complex diseases where departure from HWE was noted in either the case or control sample. In addition, we show that the new test is useful for the analysis of genome‐wide association studies. Genet. Epidemiol. 33:569–580, 2009. © 2009 Wiley‐Liss, Inc.     

The more you test,the more you find: The smallest P‐values become increasingly enriched with real findings as more tests are conducted

The increasing accessibility of data to researchers makes it possible to conduct massive amounts of statistical testing. Rather than follow specific scientific hypotheses with statistical analysis, researchers can now test many possible relationships and let statistics generate hypotheses for them. The field of genetic epidemiology is an illustrative case, where testing of candidate genetic variants for association with an outcome has been replaced by agnostic screening of the entire genome. Poor replication rates of candidate gene studies have improved dramatically with the increase in genomic coverage, due to factors such as adoption of better statistical practices and availability of larger sample sizes. Here, we suggest that another important factor behind the improved replicability of genome‐wide scans is an increase in the amount of statistical testing itself. We show that an increase in the number of tested hypotheses increases the proportion of true associations among the variants with the smallest P‐values. We develop statistical theory to quantify how the expected proportion of genuine signals (EPGS) among top hits depends on the number of tests. This enrichment of top hits by real findings holds regardless of whether genome‐wide statistical significance has been reached in a study. Moreover, if we consider only those “failed” studies that produce no statistically significant results, the same enrichment phenomenon takes place: the proportion of true associations among top hits grows with the number of tests. The enrichment occurs even if the true signals are encountered at the logarithmically decreasing rate with the additional testing.     

School-based HIV/AIDS education,risky sexual behaviors,and HIV testing among high school students in the United States

This study assessed the practical value of HIV/AIDS education among at-risk adolescents in the United States. Data were drawn from the 2013 Youth Risk Behavior Surveillance System spanning students in grades 9-12 who have engaged in sexual intercourse. A multivariate hierarchical logistic regression analysis was employed to test: (1) the individual effects of school-based HIV/AIDS education and risky sexual behaviors on the probability of HIV testing and (2) the interaction effects to estimate the degree to which the education effect varied by specific risky sexual behavior. The results indicated that students who engaged in risky sexual activities and received HIV/AIDS education were more likely to test for HIV compared to those who did not receive HIV/AIDS education. The relationship between education and HIV testing also varied according to the number of recent sexual partners. The findings have policy and practice implications. Specifically, HIV/AIDS education that promotes HIV testing should be encouraged particularly with the high-risk student population.     

A sequential testing approach to detecting multiple change points in the proportional hazards model

The semi‐parametric proportional hazards model has been widely adopted in clinical trials with time‐to‐event outcomes. A key assumption in the model is that the hazard ratio function is a constant over time, which is frequently violated as there is often a lag period before an experimental treatment reaches its full effect. One existing approach uses maximal score tests and Monte Carlo sampling to identify multiple change points in the hazard ratio function, which requires the number of change points that exist in the model to be known. We propose a sequential testing approach to detecting multiple change points in the hazard ratio function using likelihood ratio tests, and the distributions of the likelihood ratio statistics under the null hypothesis are evaluated via resampling. An important feature of the proposed approach is that the number of change points in the model is inferred from the data and does not need to be specified. Numerical results based on simulated clinical trials and a real time‐to‐event study show that the proposed approach can accurately detect the change points in the hazard ratio function. Copyright © 2012 John Wiley & Sons, Ltd.     

A test for differential ascertainment in case-control studies with application to child maltreatment

We propose a method to test for the presence of differential ascertainment in case-control studies, when data are collected by multiple sources. We show that, when differential ascertainment is present, the use of only the observed cases leads to severe bias in the computation of the odds ratio. We can alleviate the effect of such bias using the estimates that our method of testing for differential ascertainment naturally provides. We apply it to a dataset obtained from the National Violent Death Reporting System, with the goal of checking for the presence of differential ascertainment by race in the count of deaths caused by child maltreatment.     

Bootstrap confidence intervals and bias correction in the estimation of HIV incidence from surveillance data with testing for recent infection

The incidence of new infections is a key measure of the status of the HIV epidemic, but accurate measurement of incidence is often constrained by limited data. Karon et al. (Statist. Med. 2008; 27:4617–4633) developed a model to estimate the incidence of HIV infection from surveillance data with biologic testing for recent infection for newly diagnosed cases. This method has been implemented by public health departments across the United States and is behind the new national incidence estimates, which are about 40 per cent higher than previous estimates. We show that the delta method approximation given for the variance of the estimator is incomplete, leading to an inflated variance estimate. This contributes to the generation of overly conservative confidence intervals, potentially obscuring important differences between populations. We demonstrate via simulation that an innovative model-based bootstrap method using the specified model for the infection and surveillance process improves confidence interval coverage and adjusts for the bias in the point estimate. Confidence interval coverage is about 94–97 per cent after correction, compared with 96–99 per cent before. The simulated bias in the estimate of incidence ranges from ?6.3 to +14.6 per cent under the original model but is consistently under 1 per cent after correction by the model-based bootstrap. In an application to data from King County, Washington in 2007 we observe correction of 7.2 per cent relative bias in the incidence estimate and a 66 per cent reduction in the width of the 95 per cent confidence interval using this method. We provide open-source software to implement the method that can also be extended for alternate models.     

Claims on health care: a decision-making framework for equity, with application to treatment for HIV/AIDS in South Africa

Trying to determine how best to allocate resources in health care is especially difficult when resources are severely constrained, as is the case in all developing countries. This is particularly true in South Africa currently where the HIV epidemic adds significantly to a health service already overstretched by the demands made upon it. This paper proposes a framework for determining how best to allocate scarce health care resources in such circumstances. This is based on communitarian claims. The basis of possible claims considered include: the need for health care, specified both as illness and capacity to benefit; whether or not claimants have personal responsibility in the conditions that have generated their health care need; relative deprivation or disadvantage; and the impact of services on the health of society and on the social fabric. Ways of determining these different claims in practice and the weights to be attached to them are also discussed. The implications for the treatment of HIV/AIDS in South Africa are spelt out.     

A method for testing the significance of geographical correlations with application to industrial lung cancer in France

This paper discusses some of the problems encountered when testing significance in geographical epidemiology where variables typically exhibit some spatial autocorrelation. A test of partial correlation between spatially autocorrelated variables is presented. It is based on evaluation of an effective sample size which takes account of spatial structure. Performance is assessed by Monte Carlo simulation. The proposed method is used to study male lung cancer rates in specific industries in France.     

A flexible model for the mean and variance functions,with application to medical cost data

Medical cost data are often skewed to the right and heteroscedastic, having a nonlinear relation with covariates. To tackle these issues, we consider an extension to generalized linear models by assuming nonlinear associations of covariates in the mean function and allowing the variance to be an unknown but smooth function of the mean. We make no further assumption on the distributional form. The unknown functions are described by penalized splines, and the estimation is carried out using nonparametric quasi‐likelihood. Simulation studies show the flexibility and advantages of our approach. We apply the model to the annual medical costs of heart failure patients in the clinical data repository at the University of Virginia Hospital System. Copyright © 2013 John Wiley & Sons, Ltd.