A simple significance test for quantile regression

Where OLS regression seeks to model the mean of a random variable as a function of observed variables, quantile regression seeks to model the quantiles of a random variable as functions of observed variables. Tests for the dependence of the quantiles of a random variable upon observed variables have only been developed through the use of computer resampling or based on asymptotic approximations resting on distributional assumptions. We propose an exceedingly simple but heretofore undocumented likelihood ratio test within a logistic regression framework to test the dependence of a quantile of a random variable upon observed variables. Simulated data sets are used to illustrate the rationale, ease, and utility of the hypothesis test. Simulations have been performed over a variety of situations to estimate the type I error rates and statistical power of the procedure. Results from this procedure are compared to (1) previously proposed asymptotic tests for quantile regression and (2) bootstrap techniques commonly used for quantile regression inference. Results show that this less computationally intense method has appropriate type I error control, which is not true for all competing procedures, comparable power to both previously proposed asymptotic tests and bootstrap techniques, and greater computational ease. We illustrate the approach using data from 779 adolescent boys age 12-18 from the Third National Health and Nutrition Examination Survey (NHANES III) to test hypotheses regarding age, ethnicity, and their interaction upon quantiles of waist circumference.     

The impact of diagnostic error on testing genetic association in case-control studies

In case-control studies, subjects in the case group may be recruited from suspected patients who are diagnosed positively with disease. While many statistical methods have been developed for measurement error or misclassification of exposure variables in epidemiological studies, no studies have been reported on the effect of errors in diagnosing disease on testing genetic association in case-control studies. We study the impact of using the original Cochran-Armitage trend test assuming no diagnostic error when, in fact, cases and controls may be clinically diagnosed by an imperfect gold standard or a reference test. The type I error, sample size and asymptotic power of trend tests are examined under a family of genetic models in the presence of diagnostic error. The empirical powers of the trend tests are also compared by simulation studies under various genetic models.     

Combining stratified and unstratified log‐rank tests in paired survival data

The log‐rank test is the most widely used nonparametric method for testing treatment differences in survival between two treatment groups due to its efficiency under the proportional hazards model. Most previous work on the log‐rank test has assumed that the samples from the two treatment groups are independent. This assumption is not always true. In multi‐center clinical trials, survival times of patients in the same medical center may be correlated due to factors specific to each center. For such data, we can construct both stratified and unstratified log‐rank tests. These two tests turn out to have very different powers for correlated samples. An appropriate linear combination of these two tests may give a more powerful test than either of the individual test. Under a bivariate frailty model, we obtain closed‐form asymptotic local alternative distributions and the correlation coefficient between these two tests. Based on these results we construct an optimal linear combination of the two test statistics to maximize the local power. Simulation studies with Hougaard's model confirm our construction. We also study the robustness of the combined test by simulations. Copyright © 2010 John Wiley & Sons, Ltd.     

Exact unconditional tests for testing non-inferiority in matched-pairs design

The problem of testing non-inferiority in a 2 x 2 matched-pairs sample is considered. Two exact unconditional tests based on the standard and the confidence interval p-values are proposed. Although tests of non-inferiority have two nuisance parameters under the null hypothesis, the exact tests are defined by reducing the dimension of nuisance parameter space from two to one using the monotonicity of the distribution. The exact sizes and powers of these tests and the existing asymptotic test are considered. The exact tests are found to be accurate in view of their size property. In addition, the exact test based on the confidence interval p-value is more powerful than the other exact test. It is shown that the asymptotic test is inaccurate, that is, its size exceeds the claimed nominal level alpha. Therefore, it recommends a cautious approach in use of the asymptotic test for the problem of testing non-inferiority, particularly when sample sizes are small or moderately large.     

Sample size determination for comparing several survival curves with unequal allocations

Ahnn and Anderson derived sample size formulae for unstratified and stratified designs assuming equal allocation of subjects to three or more treatment groups. We generalize the sample size formulae to allow for unequal allocation. In addition, we define the overall probability of death to be equal to one minus the censored proportion for the stratified design. This definition also leads to a slightly different definition of the non-centrality parameter than that of Ahnn and Anderson for the stratified case. Assuming proportional hazards, sample sizes are determined for a prespecified power, significance level, hazard ratios, allocation of subjects to several treatment groups, and known censored proportion. In the proportional hazards setting, three cases are considered: (1) exponential failures--exponential censoring, (2) exponential failures--uniform censoring, and (3) Weibull failures (assuming same shape parameter for all groups)--uniform censoring. In all three cases of the unstratified case, it is assumed that the censoring distribution is the same for all of the treatment groups. For the stratified log-rank test, it is assumed the same censoring distribution across the treatment groups and the strata. Further, formulae have been developed to provide approximate powers for the test, based upon the first two or first four-moments of the asymptotic distribution. We observe the following two major findings based on the simulations. First, the simulated power of the log-rank test does not depend on the censoring mechanism. Second, for a significance level of 0.05 and power of 0.80, the required sample size n is independent of the censoring pattern. Moreover, there is very close agreement between the exact (asymptotic) and simulated powers when a sequence of alternatives is close to the null hypothesis. Two-moment and four-moment power series approximations also yield powers in close agreement with the exact (asymptotic) power. With unequal allocations, our simulations show that the empirical powers are consistently above the target value of prespecified power of 0.80 when 50 per cent of the patients are allocated to the treatment group with the smallest hazard.     

Permutation and Bayesian tests for testing random effects in linear mixed-effects models

In many applications of linear mixed-effects models to longitudinal and multilevel data especially from medical studies, it is of interest to test for the need of random effects in the model. It is known that classical tests such as the likelihood ratio, Wald, and score tests are not suitable for testing random effects because they suffer from testing on the boundary of the parameter space. Instead, permutation and bootstrap tests as well as Bayesian tests, which do not rely on the asymptotic distributions, avoid issues with the boundary of the parameter space. In this paper, we first develop a permutation test based on the likelihood ratio test statistic, which can be easily used for testing multiple random effects and any subset of them in linear mixed-effects models. The proposed permutation test would be an extension to two existing permutation tests. We then aim to compare permutation tests and Bayesian tests for random effects to find out which test is more powerful under which situation. Nothing is known about this in the literature, although this is an important practical problem due to the usefulness of both methods in tackling the challenges with testing random effects. For this, we consider a Bayesian test developed using Bayes factors, where we also propose a new alternative computation for this Bayesian test to avoid some computational issue it encounters in testing multiple random effects. Extensive simulations and a real data analysis are used for evaluation of the proposed permutation test and its comparison with the Bayesian test. We find that both tests perform well, albeit the permutation test with the likelihood ratio statistic tends to provide a relatively higher power when testing multiple random effects.     

Likelihood ratio and score tests to test the non‐inferiority (or equivalence) of the odds ratio in a crossover study with binary outcomes

We consider the non‐inferiority (or equivalence) test of the odds ratio (OR) in a crossover study with binary outcomes to evaluate the treatment effects of two drugs. To solve this problem, Lui and Chang (2011) proposed both an asymptotic method and a conditional method based on a random effects logit model. Kenward and Jones (1987) proposed a likelihood ratio test (LRT_M) based on a log linear model. These existing methods are all subject to model misspecification. In this paper, we propose a likelihood ratio test (LRT) and a score test that are independent of model specification. Monte Carlo simulation studies show that, in scenarios considered in this paper, both the LRT and the score test have higher power than the asymptotic and conditional methods for the non‐inferiority test; the LRT, score, and asymptotic methods have similar power, and they all have higher power than the conditional method for the equivalence test. When data can be well described by a log linear model, the LRT_M has the highest power among all the five methods (LRT_M, LRT, score, asymptotic, and conditional) for both non‐inferiority and equivalence tests. However, in scenarios for which a log linear model does not describe the data well, the LRT_M has the lowest power for the non‐inferiority test and has inflated type I error rates for the equivalence test. We provide an example from a clinical trial that illustrates our methods. Copyright © 2016 John Wiley & Sons, Ltd.     

Tests for equivalence or non-inferiority for paired binary data.

Assessment of therapeutic equivalence or non-inferiority between two medical diagnostic procedures often involves comparisons of the response rates between paired binary endpoints. The commonly used and accepted approach to assessing equivalence is by comparing the asymptotic confidence interval on the difference of two response rates with some clinical meaningful equivalence limits. This paper investigates two asymptotic test statistics, a Wald-type (sample-based) test statistic and a restricted maximum likelihood estimation (RMLE-based) test statistic, to assess equivalence or non-inferiority based on paired binary endpoints. The sample size and power functions of the two tests are derived. The actual type I error and power of the two tests are computed by enumerating the exact probabilities in the rejection region. The results show that the RMLE-based test controls type I error better than the sample-based test. To establish an equivalence between two treatments with a symmetric equivalence limit of 0.15, a minimal sample size of 120 is needed. The RMLE-based test without the continuity correction performs well at the boundary point 0. A numerical example illustrates the proposed procedures.     

Test non-inferiority (and equivalence) based on the odds ratio under a simple crossover trial

For testing the non-inferiority (or equivalence) of a generic drug to a standard drug, the odds ratio (OR) of patient response rates has been recommended to measure the relative treatment efficacy. On the basis of a random effects logistic regression model, we develop asymptotic test procedures for testing non-inferiority and equivalence with respect to the OR of patient response rates under a simple crossover design. We further derive exact test procedures, which are especially useful for the situations in which the number of patients in a crossover trial is small. We address sample size calculation for testing non-inferiority and equivalence based on the asymptotic test procedures proposed here. We also discuss estimation of the OR of patient response rates for both the treatment and period effects. Finally, we include two examples, one comparing two solution aerosols in treating asthma, and the other one studying two inhalation devices for asthmatics, to illustrate the use of the proposed test procedures and estimators.     

An exact maternal-fetal genotype incompatibility (MFG) test

The maternal-fetal genotype incompatibility (MFG) test can be used for a variety of genetic applications concerning disease risk in offspring including testing for the presence of alleles that act directly through offspring genotypes (child allelic effects), alleles that act through maternal genotypes (maternal allelic effects), or maternal-fetal genotype incompatibilities. The log-linear version of the MFG model divides the genotype data into many cells, where each cell represents one of the possible mother, father, and child genotype combinations. Currently, tests of hypotheses about different allelic effects are accomplished by an asymptotic MFG test, but it is unknown if this is appropriate under conditions that produce small cell counts. In this report, we develop an exact MFG test that is based on the permutation distribution of cell counts. We determine by simulation the type I error and power of both the exact MFG test and the asymptotic MFG test for four different biologically relevant scenarios: a test of child allelic effects in the presence of maternal allelic effects, a test of maternal allelic effects in the presence of child allelic effects, and tests of maternal-fetal genotype incompatibility with and without child allelic effects. These simulations show that, in general, the exact test is slightly conservative whereas the asymptotic test is slightly anti-conservative. However, the asymptotic MFG test produces significantly inflated type I error rates under conditions with extreme null allele frequencies and sample sizes of 75, 100, and 150. Under these conditions, the exact test is clearly preferred over the asymptotic test. Under all other conditions that we tested, the user can safely choose either the exact test or the asymptotic test.     

Exact equivalence test for risk ratio and its sample size determination under inverse sampling

When data are dichotomous, this paper notes the utility of inverse sampling in establishing equivalence with respect to the risk ratio. This paper develops an exact equivalence test that accounts for the risk ratio under inverse sampling and further discusses the relationship between the exact equivalence test and the exact conditional confidence limits. Also included are an exact and two asymptotic procedures for calculation of the minimum required number of index subjects for a desired power 1−β at a given α-level. Finally, this paper provides a table that summarizes the minimum required number of index subjects for powers equal to 0⋅90 and 0⋅80 in application of the proposed exact equivalence test at 0⋅05-level in a variety of situations. © 1997 by John Wiley & Sons, Ltd.     

The use of an extended baseline period in the evaluation of treatment in a longitudinal Duchenne muscular dystrophy trial

A trial of Duchenne muscular dystrophy involved tracking boys of all ages through a one-year baseline period, followed by a one-year trial of leucine versus placebo treatment. In this paper we develop a model for a total-muscle-strength score that uses the data of the extended baseline period in the evaluation of the leucine treatment. The model is based on a polynomial growth curve in age whose coefficients can vary according to treatment or phase. Maximum likelihood estimates of the parameters of the model are obtained from use of the EM algorithm. We propose tests for the adequacy of the model as well as for treatment effects. A quadratic model appears the most parsimonious fit to the data and there is no evidence of any leucine effect on scores. We examine the asymptotic power of the test for treatment effect and compare it with that of a simpler analysis.     

Evaluation by simulation of tests based on non-linear mixed-effects models in pharmacokinetic interaction and bioequivalence cross-over trials

We propose tests based on non-linear mixed effects models (NLMEM) in pharmacokinetic interaction and bioequivalence cross-over trials comparing two treatments or two formulations. To compare the logarithm of the area under the curve (AUC) using these models, two approaches are studied: in the first one, concentration data are analysed globally, with and without the estimation of a treatment effect; and in the second one, they are analysed separately in each treatment group with the estimation of the individual parameters. Four tests for comparison of the logarithm AUC between two treatment arms are studied: a likelihood-ratio test (LRT), a Wald test and two tests, parametric and non-parametric, comparing the individual Empirical Bayes (EB) estimates. These tests are adapted to the case of equivalence, except the LRT which does not have any simple extension. We evaluate by simulation of the type I error and the power for both comparison and equivalence tests. They are compared to the standard tests recommended by the FDA and the EMEA, based on non-compartmental (NC) AUC. Trials for a usual PK model are simulated under H(0) and several H(1) using S-plus software and analysed with the nlme function. Different configurations of the number of subjects (n=12, 24 and 40) and of the number of samples per subject (J=10, 5 and 3) are studied. The type I error alpha of LRT and Wald comparison test in the 5000 replications of interaction cross-over trials is found to be 20.9 per cent and 21.7 per cent, respectively, in the original design (n=12, J=10), which is far superior to 5 per cent, and decreases when n increases. When n is fixed, alpha is found to increase with J. Power is satisfactory for both tests, after correction of the significance threshold. Results of EB and NC tests are similar with satisfactory powers and a type I error close to 5 per cent, except when J=3 for EB tests. Similar results are obtained for equivalence tests, except for EB and NC Student tests, which are not of a great interest. NC tests keep their place when the number of samples per subject J is large, but NLMEM seem useful for cross-over studies performed in special populations where J limited; the evaluation by Monte-Carlo simulations of empirical threshold seems however necessary because of the inflation of the type I error.     

Evaluation of old and new tests of heterogeneity in epidemiologic meta-analysis.

The identification of heterogeneity in effects between studies is a key issue in meta-analyses of observational studies, since it is critical for determining whether it is appropriate to pool the individual results into one summary measure. The result of a hypothesis test is often used as the decision criterion. In this paper, the authors use a large simulation study patterned from the key features of five published epidemiologic meta-analyses to investigate the type I error and statistical power of five previously proposed asymptotic homogeneity tests, a parametric bootstrap version of each of the tests, and tau2-bootstrap, a test proposed by the authors. The results show that the asymptotic DerSimonian and Laird Q statistic and the bootstrap versions of the other tests give the correct type I error under the null hypothesis but that all of the tests considered have low statistical power, especially when the number of studies included in the meta-analysis is small (<20). From the point of view of validity, power, and computational ease, the Q statistic is clearly the best choice. The authors found that the performance of all of the tests considered did not depend appreciably upon the value of the pooled odds ratio, both for size and for power. Because tests for heterogeneity will often be underpowered, random effects models can be used routinely, and heterogeneity can be quantified by means of R(I), the proportion of the total variance of the pooled effect measure due to between-study variance, and CV(B), the between-study coefficient of variation.     

Exact and asymptotic tests for homogeneity in several 2 x 2 tables.

This paper presents the results of a Monte Carlo study comparing the performance, in terms of size and power, of six exact and six asymptotic tests for the homogeneity of odds ratios in several 2 x 2 contingency tables. With a small sample size or sparse data structure, the exact tests performed better than the asymptotic tests because they maintained the nominal size and, in some situations, had slightly higher power. Among the exact tests, we recommend the Zelen, Pearson chi-square and scores tests. Among the asymptotic tests, the Breslow-Day and Pearson chi-square tests were slightly better in some situations than the unconditional and conditional score tests. However, both exact and asymptotic tests had low power for small strata sizes, even with moderate to large heterogeneity of odds ratios. Corroborating previous findings, the asymptotic unconditional likelihood ratio test was too liberal in terms of size.     

Effect of unequal censoring on the size and power of the logrank and Wilcoxon types of tests for survival data

This paper examines the effect of random unequal censoring on the size and power of two-sample logrank and Wilcoxon types of tests for comparing two survival distributions by simulation with small samples from censored exponential distributions. We compared equal-sized samples of n = 8, 16, and 32 with 1000 (size) and 500 (power) simulation trials for 16 combinations of the censoring proportions of 0, 20, 40, and 60 per cent in each of the two samples. For n = 8, the asymptotic normality (AN), Peto-Peto, and the two Wilcoxon-type tests performed at nominal 5 per cent size expectations, but the Mantel test exceeded the 5 per cent size acceptance region in 6 of 16 censoring combinations. For n = 16 and 32, all tests showed proper size, with the Peto-Peto test being most conservative in the presence of unequal censoring. We compared powers of all tests for exponential hazard ratios of 1·4 and 2·0. The Mantel test showed 90 to 95 per cent power efficiency relative to the parametric AN test. Both Wilcoxon tests performed identically and had the lowest relative power of all tests examined but appeared most robust to the differential censoring patterns studied. A modified version of the Peto-Peto test showed power comparable to the Mantel test.     

Sample size and power calculations for correlations between bivariate longitudinal data

The analysis of a baseline predictor with a longitudinally measured outcome is well established and sample size calculations are reasonably well understood. Analysis of bivariate longitudinally measured outcomes is gaining in popularity and methods to address design issues are required. The focus in a random effects model for bivariate longitudinal outcomes is on the correlations that arise between the random effects and between the bivariate residuals. In the bivariate random effects model, we estimate the asymptotic variances of the correlations and we propose power calculations for testing and estimating the correlations. We compare asymptotic variance estimates to variance estimates obtained from simulation studies and compare our proposed power calculations for correlations on bivariate longitudinal data to power calculations for correlations on cross‐sectional data. Copyright © 2010 John Wiley & Sons, Ltd.     

An approximate unconditional test of non-inferiority between two proportions

This paper investigates an approximate unconditional test for non-inferiority between two independent binomial proportions. The P-value of the approximate unconditional test is evaluated using the maximum likelihood estimate of the nuisance parameter. In this paper, we clarify some differences in defining the rejection regions between the approximate unconditional and conventional conditional or unconditional exact test. We compare the approximate unconditional test with the asymptotic test and unconditional exact test by Chan (Statistics in Medicine, 17, 1403-1413, 1998) with respect to the type I error and power. In general, the type I errors and powers are in the decreasing order of the asymptotic, approximate unconditional and unconditional exact tests. In many cases, the type I errors are above the nominal level from the asymptotic test, and are below the nominal level from the unconditional exact test. In summary, when the non-inferiority test is formulated in terms of the difference between two proportions, the approximate unconditional test is the most desirable, because it is easier to implement and generally more powerful than the unconditional exact test and its size rarely exceeds the nominal size. However, when a test between two proportions is formulated in terms of the ratio of two proportions, such as a test of efficacy, more caution should be made in selecting a test procedure. The performance of the tests depends on the sample size and the range of plausible values of the nuisance parameter. Published in 2000 by John Wiley & Sons, Ltd.