A more powerful exact test of noninferiority from binary matched-pairs data

Assessing the therapeutic noninferiority of one medical treatment compared with another is often based on the difference in response rates from a matched binary pairs design. This paper develops a new exact unconditional test for noninferiority that is more powerful than available alternatives. There are two new elements presented in this paper. First, we introduce the likelihood ratio statistic as an alternative to the previously proposed score statistic of Nam (Biometrics 1997; 53:1422-1430). Second, we eliminate the nuisance parameter by estimation followed by maximization as an alternative to the partial maximization of Berger and Boos (Am. Stat. Assoc. 1994; 89:1012-1016) or traditional full maximization. Based on an extensive numerical study, we recommend tests based on the score statistic, the nuisance parameter being controlled by estimation followed by maximization.     

An approximate unconditional test of non-inferiority between two proportions

This paper investigates an approximate unconditional test for non-inferiority between two independent binomial proportions. The P-value of the approximate unconditional test is evaluated using the maximum likelihood estimate of the nuisance parameter. In this paper, we clarify some differences in defining the rejection regions between the approximate unconditional and conventional conditional or unconditional exact test. We compare the approximate unconditional test with the asymptotic test and unconditional exact test by Chan (Statistics in Medicine, 17, 1403-1413, 1998) with respect to the type I error and power. In general, the type I errors and powers are in the decreasing order of the asymptotic, approximate unconditional and unconditional exact tests. In many cases, the type I errors are above the nominal level from the asymptotic test, and are below the nominal level from the unconditional exact test. In summary, when the non-inferiority test is formulated in terms of the difference between two proportions, the approximate unconditional test is the most desirable, because it is easier to implement and generally more powerful than the unconditional exact test and its size rarely exceeds the nominal size. However, when a test between two proportions is formulated in terms of the ratio of two proportions, such as a test of efficacy, more caution should be made in selecting a test procedure. The performance of the tests depends on the sample size and the range of plausible values of the nuisance parameter. Published in 2000 by John Wiley & Sons, Ltd.     

Exact unconditional tests for testing non-inferiority in matched-pairs design

The problem of testing non-inferiority in a 2 x 2 matched-pairs sample is considered. Two exact unconditional tests based on the standard and the confidence interval p-values are proposed. Although tests of non-inferiority have two nuisance parameters under the null hypothesis, the exact tests are defined by reducing the dimension of nuisance parameter space from two to one using the monotonicity of the distribution. The exact sizes and powers of these tests and the existing asymptotic test are considered. The exact tests are found to be accurate in view of their size property. In addition, the exact test based on the confidence interval p-value is more powerful than the other exact test. It is shown that the asymptotic test is inaccurate, that is, its size exceeds the claimed nominal level alpha. Therefore, it recommends a cautious approach in use of the asymptotic test for the problem of testing non-inferiority, particularly when sample sizes are small or moderately large.     

Exact approaches for testing hypotheses based on the intra‐class kappa coefficient

Testing involving the intra‐class kappa coefficient is commonly performed in order to assess agreement involving categorical ratings. A number of procedures have been proposed, which make use of the limiting null distribution as the sample size goes to infinity in order to compute the observed significance. As with many tests based on asymptotic null distributions, these tests are associated with problematic type I error control for selected sample sizes and points in the parameter space. We propose and study a collection of exact testing approaches for both the one‐sample and K‐sample scenarios. For the one‐sample case, p‐values are obtained using the exact distribution of the test statistic conditional on a sufficient statistic. In addition, unconditional approaches are considered on the basis of maximization across the nuisance parameter space. Numerical evaluation reveals advantages with the exact unconditional procedures. Copyright © 2014 John Wiley & Sons, Ltd.     

An efficient and exact approach for detecting trends with binary endpoints

Lloyd (Aust. Nz. J. Stat., 50, 329-345, 2008) developed an exact testing approach to control for nuisance parameters, which was shown to be advantageous in testing for differences between two population proportions. We utilized this approach to obtain unconditional tests for trends in 2 × K contingency tables. We compare the unconditional procedure with other unconditional and conditional approaches based on the well-known Cochran-Armitage test statistic. We give an example to illustrate the approach, and provide a comparison between the methods with regards to type I error and power. The proposed procedure is preferable because it is less conservative and has superior power properties.     

Exact and asymptotic tests for homogeneity in several 2 x 2 tables.

This paper presents the results of a Monte Carlo study comparing the performance, in terms of size and power, of six exact and six asymptotic tests for the homogeneity of odds ratios in several 2 x 2 contingency tables. With a small sample size or sparse data structure, the exact tests performed better than the asymptotic tests because they maintained the nominal size and, in some situations, had slightly higher power. Among the exact tests, we recommend the Zelen, Pearson chi-square and scores tests. Among the asymptotic tests, the Breslow-Day and Pearson chi-square tests were slightly better in some situations than the unconditional and conditional score tests. However, both exact and asymptotic tests had low power for small strata sizes, even with moderate to large heterogeneity of odds ratios. Corroborating previous findings, the asymptotic unconditional likelihood ratio test was too liberal in terms of size.     

Recommended tests for association in 2×2 tables

The asymptotic Pearson's chi‐squared test and Fisher's exact test have long been the most used for testing association in 2×2 tables. Unconditional tests preserve the significance level and generally are more powerful than Fisher's exact test for moderate to small samples, but previously were disadvantaged by being computationally demanding. This disadvantage is now moot, as software to facilitate unconditional tests has been available for years. Moreover, Fisher's exact test with mid‐p adjustment gives about the same results as an unconditional test. Consequently, several better tests are available, and the choice of a test should depend only on its merits for the application involved. Unconditional tests and the mid‐p approach ought to be used more than they now are. The traditional Fisher's exact test should practically never be used. Copyright © 2009 John Wiley & Sons, Ltd.     

完全随机设计两样本率比较的非条件确切检验方法

目的 探讨完全随机设计两样本率比较的Bamard确切非条件检验方法，并与Fishe，确切概率法进行比较。方法 非条件确切检验方法是通过使冗余参数在其参数空间内变动，以其确切P值的上确界为所求检验P值。结果 根据实例分别算得两种方法的确切分布与P值，Bamard方法所得P值较Fisher方法更小。结论 在小样本情形下。Barnard确切非条件检验法较Fisher确切条件检验法获得的检验P值更小，检验功效更高。     

Exact unconditional tables for significance testing in the 2 x 2 multinomial trial.

This paper presents tables analogous to T-tables for use in the 2 x 2 multinomial trial, where the continuity corrected Z-statistic is used to make exact unconditional inference. This is the first solution of a discrete exact unconditional inference problem involving a multivariate nuisance parameter for which no ancillary statistic exists.     

Equivalence of three score tests for association mapping of quantitative trait loci under selective genotyping

Huang and Lin ([2007] Am J Hum Genet 80:567–572) proposed a conditional‐likelihood approach for mapping quantitative trait loci (QTL) under selective genotyping, and demonstrated via simulation that their model tends to be more powerful than the prospective linear regression. However, we show that the three score tests based on the conditional, prospective and retrospective likelihoods are numerically identical in testing association between a quantitative trait and a candidate locus. Two approximations are derived for calculating power and sample size for the score test. Compared to the random sampling, a single‐tail selection generally reduces the power of the score test in mapping small effect QTLs. A two‐tail selection generally enhances the QTL heritability; however, in small samples, the power of the test may actually decrease if the sample sizes are highly unbalanced in the upper and lower tails of the trait distribution. Genet. Epidemiol. 34: 522–527, 2010. © 2010 Wiley‐Liss, Inc.     

Choice of test for association in small sample unordered r x c tables

Pearson's chi-squared, the likelihood-ratio, and Fisher-Freeman-Halton's test statistics are often used to test the association of unordered r x c tables. Asymptotical, exact conditional, or exact conditional with mid-p adjustment methods are commonly used to compute the p-value. We have compared test power and significance level for these test statistics and p-value calculations in small sample r x c tables, mostly 3 x 2 and some with both r and c are greater than 2. After extensive simulations, in general we recommend using an exact conditional mid-p test with Pearson's chi-squared or Fisher-Freeman-Halton's statistic, which usually is the most powerful test yet preserve the approximate significance level. Moreover, we recommend that the asymptotic Pearson's chi-squared or other asymptotic tests not be used for small sample r x c tables.     

Adaptive extensions of a two-stage group sequential procedure for testing primary and secondary endpoints (I): unknown correlation between the endpoints

In a previous paper we studied a two‐stage group sequential procedure (GSP) for testing primary and secondary endpoints where the primary endpoint serves as a gatekeeper for the secondary endpoint. We assumed a simple setup of a bivariate normal distribution for the two endpoints with the correlation coefficient ρ between them being either an unknown nuisance parameter or a known constant. Under the former assumption, we used the least favorable value of ρ = 1 to compute the critical boundaries of a conservative GSP. Under the latter assumption, we computed the critical boundaries of an exact GSP. However, neither assumption is very practical. The ρ = 1 assumption is too conservative resulting in loss of power, whereas the known ρ assumption is never true in practice. In this part I of a two‐part paper on adaptive extensions of this two‐stage procedure (part II deals with sample size re‐estimation), we propose an intermediate approach that uses the sample correlation coefficient r from the first‐stage data to adaptively adjust the secondary boundary after accounting for the sampling error in r via an upper confidence limit on ρ by using a method due to Berger and Boos. We show via simulation that this approach achieves 5–11% absolute secondary power gain for ρ ≤0.5. The preferred boundary combination in terms of high primary as well as secondary power is that of O'Brien and Fleming for the primary and of Pocock for the secondary. The proposed approach using this boundary combination achieves 72–84% relative secondary power gain (with respect to the exact GSP that assumes known ρ). We give a clinical trial example to illustrate the proposed procedure. Copyright © 2012 John Wiley & Sons, Ltd.     

Efficient power computation for exact and mid-P tests for the common odds ratio in several 2 × 2 tables

When designing a study that may generate a set of sparse 2 × 2 tables, or when confronted with ‘negative’ results upon exact analysis of such tables, we need to compute the power of exact tests. In this paper we provide an efficient approach for computing exact unconditional power for four exact tests on the common odds ratio in a series of 2 × 2 tables. These tests are the traditional exact test; a test based on a probability ordering of the sample space; and two tests based on ordering the sample space according to distance from the mean, or median. For each test, we consider both a conservative version and a mid-P adjusted version. We explore three computational options for power determination: exact power computation, calculation of exact upper and lower bounds for power, and Monte Carlo confidence bounds for power. We present an interactive program implementing these options. For study design, the program may be run several times to arrive at a sample configuration with adequate power.     

Statistical approaches to establishing vaccine safety

In a vaccine safety trial, the primary interest is to demonstrate that the vaccine is sufficiently safe, rejecting the null hypothesis that the relative risk of an adverse event attributable to the new vaccine is above a prespecified value, greater than one. We evaluate the exact probability of type I error of the likelihood score test, with sample size determined by normal approximation, by enumeration of the binomial outcomes in the rejection region and show that it exceeds the nominal level. In the case of rare adverse events, we recommend the Poisson approximation as an alternative and develop the corresponding conditional and unconditional tests. We give sample size and power calculations for these tests. We also propose optimal randomization strategies which either (i) minimize the total number of adverse cases or (ii) minimize the expected number of subjects when the vaccine is unsafe. We illustrate the proposed methods using a hypothetical vaccine safety study.     

On the use of the generalized t and generalized rank-sum statistics in medical research.

We have used Monte Carlo methods to compare the type I error properties of the conditional and unconditional versions of the generalized t and the generalized rank-sum tests to those of the independent samples t and Wilcoxon rank-sum tests. Results showed inflated type I errors for the conditional generalized tests but not for the unconditional tests. We also compared the power of the unconditional generalized tests to that of the t and Wilcoxon tests under a variety of conditions. Results showed the generalized tests to be much more efficient than their traditional counterparts in some circumstances, but substantially less powerful in others. Based on these and other considerations, we conclude that the application of these newer statistics in medical research needs further consideration.     

Joint analysis of multiple phenotypes for extremely unbalanced case-control association studies

In genome-wide association studies (GWAS) for thousands of phenotypes in biobanks, most binary phenotypes have substantially fewer cases than controls. Many widely used approaches for joint analysis of multiple phenotypes produce inflated type I error rates for such extremely unbalanced case-control phenotypes. In this research, we develop a method to jointly analyze multiple unbalanced case-control phenotypes to circumvent this issue. We first group multiple phenotypes into different clusters based on a hierarchical clustering method, then we merge phenotypes in each cluster into a single phenotype. In each cluster, we use the saddlepoint approximation to estimate the p value of an association test between the merged phenotype and a single nucleotide polymorphism (SNP) which eliminates the issue of inflated type I error rate of the test for extremely unbalanced case-control phenotypes. Finally, we use the Cauchy combination method to obtain an integrated p value for all clusters to test the association between multiple phenotypes and a SNP. We use extensive simulation studies to evaluate the performance of the proposed approach. The results show that the proposed approach can control type I error rate very well and is more powerful than other available methods. We also apply the proposed approach to phenotypes in category IX (diseases of the circulatory system) in the UK Biobank. We find that the proposed approach can identify more significant SNPs than the other viable methods we compared with.     

On the analysis of very small samples of Gaussian repeated measurements: an alternative approach

The analysis of very small samples of Gaussian repeated measurements can be challenging. First, due to a very small number of independent subjects contributing outcomes over time, statistical power can be quite small. Second, nuisance covariance parameters must be appropriately accounted for in the analysis in order to maintain the nominal test size. However, available statistical strategies that ensure valid statistical inference may lack power, whereas more powerful methods may have the potential for inflated test sizes. Therefore, we explore an alternative approach to the analysis of very small samples of Gaussian repeated measurements, with the goal of maintaining valid inference while also improving statistical power relative to other valid methods. This approach uses generalized estimating equations with a bias‐corrected empirical covariance matrix that accounts for all small‐sample aspects of nuisance correlation parameter estimation in order to maintain valid inference. Furthermore, the approach utilizes correlation selection strategies with the goal of choosing the working structure that will result in the greatest power. In our study, we show that when accurate modeling of the nuisance correlation structure impacts the efficiency of regression parameter estimation, this method can improve power relative to existing methods that yield valid inference. Copyright © 2017 John Wiley & Sons, Ltd.     

Exact and approximate unconditional confidence intervals for proportion difference in the presence of incomplete data

Confidence interval (CI) construction with respect to proportion/rate difference for paired binary data has become a standard procedure in many clinical trials and medical studies. When the sample size is small and incomplete data are present, asymptotic CIs may be dubious and exact CIs are not yet available. In this article, we propose exact and approximate unconditional test‐based methods for constructing CI for proportion/rate difference in the presence of incomplete paired binary data. Approaches based on one‐ and two‐sided Wald's tests will be considered. Unlike asymptotic CI estimators, exact unconditional CI estimators always guarantee their coverage probabilities at or above the pre‐specified confidence level. Our empirical studies further show that (i) approximate unconditional CI estimators usually yield shorter expected confidence width (ECW) with their coverage probabilities being well controlled around the pre‐specified confidence level; and (ii) the ECWs of the unconditional two‐sided‐test‐based CI estimators are generally narrower than those of the unconditional one‐sided‐test‐based CI estimators. Moreover, ECWs of asymptotic CIs may not necessarily be narrower than those of two‐sided‐based exact unconditional CIs. Two real examples will be used to illustrate our methodologies. Copyright © 2008 John Wiley & Sons, Ltd.     

On simultaneous assessment of sensitivity and specificity when combining two diagnostic tests

Diagnostic tests are seldom adopted in isolation. Few tests have high sensitivity and specificity simultaneously. In these cases, one can increase either the sensitivity or the specificity by combining two component tests under either the 'either positive' rule or the 'both positive' rule. However, there is a tradeoff between sensitivity and specificity when these rules are applied. We propose three statistical procedures to simultaneously assess the sensitivity and specificity when combining two component tests. Measurements of interest include rate difference and rate ratio. Our empirical results demonstrate that (i) the asymptotic test procedures for both measurements and approximate test procedure for rate difference possess inflated type I error rate; (ii) the exact test procedures for both measurements possess deflated type I error rate; and (iii) the approximate (unconditional) test procedure for rate ratio becomes an reliable alternative and nicely controls the actual type I error rate in small to moderate sample sizes. Moreover, the approximate (unconditional) test procedure is computationally less intensive than the exact (unconditional) test procedure. We illustrate our methodologies with a real example from a residual nasopharyngeal carcinoma (RNP) study.     

Power and sample size evaluation for the McNemar test with application to matched case-control studies.

Various expressions have appeared for sample size calculation based on the power function of McNemar's test for paired or matched proportions, especially with reference to a matched case-control study. These differ principally with respect to the expression for the variance of the statistic under the alternative hypothesis. In addition to the conditional power function, I identify and compare four distinct unconditional expressions. I show that the unconditional calculation of Schlesselman for the matched case-control study can be expressed as a first-order unconditional calculation as described by Miettinen. Corrections to Schlesselman's unconditional expression presented by Fleiss and Levin and by Dupont, which use different models to describe exposure association among matched cases and controls, are also equivalent to a first-order unconditional calculation. I present a simplification of these corrections that directly provides the underlying table of cell probabilities, from which one can perform any of the alternative sample size calculations. Also, I compare the four unconditional sample size expressions relative to the exact power function. The conclusion is that Miettinen's first-order expression tends to underestimate sample size, while his second-order expression is usually fairly accurate, though possibly slightly anti-conservative. A multinomial-based expression presented by Connor, among others, is also fairly accurate and is usually slightly conservative. Finally, a local unconditional expression of Mitra, among others, tends to be excessively conservative.