Goodness-of-fit tests for ordinal response regression models

It is well documented that the commonly used Pearson chi-square and deviance statistics are not adequate for assessing goodness-of-fit in logistic regression models when continuous covariates are modelled. In recent years, several methods have been proposed which address this shortcoming in the binary logistic regression setting or assess model fit differently. However, these techniques have typically not been extended to the ordinal response setting and few techniques exist to assess model fit in that case. We present the modified Pearson chi-square and deviance tests that are appropriate for assessing goodness-of-fit in ordinal response models when both categorical and continuous covariates are present. The methods have good power to detect omitted interaction terms and reasonable power to detect failure of the proportional odds assumption or modelling the wrong functional form of a continuous covariate. These tests also provide immediate information as to where a model may not fit well. In addition, the methods are simple to understand and implement, and are non-specific. That is, they do not require prespecification of a type of lack-of-fit to detect.     

A goodness‐of‐fit test for the ordered stereotype model

This paper presents a new goodness‐of‐fit test for an ordered stereotype model used for an ordinal response variable. The proposed test is based on the well‐known Hosmer–Lemeshow test and its version for the proportional odds regression model. The latter test statistic is calculated from a grouping scheme assuming that the levels of the ordinal response are equally spaced which might be not true. One of the main advantages of the ordered stereotype model is that it allows us to determine a new uneven spacing of the ordinal response categories, dictated by the data. The proposed test takes the use of this new adjusted spacing to partition data. A simulation study shows good performance of the proposed test under a variety of scenarios. Finally, the results of the application in two examples are presented. Copyright © 2016 John Wiley & Sons, Ltd.     

多分类 logistic 回归在冠心病危险因素研究中的应用

目的：应用多分类logistic回归找出冠心病危险因素与冠状动脉病变程度间的关系,建立冠状动脉病变程度危险因素的“最优”回归方程。方法采取系统抽样的方法,对来自2005年1月至2009年12月期间山东省潍坊市某几所医院心血管内科初步诊断为冠心病并进行冠脉造影的患者病例,抽取256例作为多分类logistic回归分析的样本。冠心病组根据狭窄病变累及血管范围分为1支病变（单支病变组）、2支病变（双支病变组）和3支及以上病变（多支病变组）。通过多分类logistic回归方法分析冠心病与冠脉病变程度相关关系。结果以冠脉病变程度为因变量,各个因素为自变量。根据单因素分析的结果,从20个研究因素中筛选出8个有统计学意义的影响因素,经过数据相关性分析、共线性诊断及专业解释等,筛选出8个影响因素进入多分类logistic回归中,最后得到与冠脉病变程度有关的有统计学意义的影响因素6个,分别是年龄、合并疾病、心率、血糖、脂蛋白（a）;保护因素1个：X17（载脂蛋白A1）。根据筛选出的6个影响因素建立“最优”回归方程。结论应用多分类logistic回归找出与冠脉病变程度有关的危险因素,并定量分析出各危险因素在冠脉病变不同程度上的概率值。     

Proportional‐odds models for repeated composite and long ordinal outcome scales

In many medical studies, researchers widely use composite or long ordinal scores, that is, scores that have a large number of categories and a natural ordering often resulting from the sum of a number of short ordinal scores, to assess function or quality of life. Typically, we analyse these using unjustified assumptions of normality for the outcome measure, which are unlikely to be even approximately true. Scores of this type are better analysed using methods reserved for more conventional (short) ordinal scores, such as the proportional‐odds model. We can avoid the need for a large number of cut‐point parameters that define the divisions between the score categories for long ordinal scores in the proportional‐odds model by the inclusion of orthogonal polynomial contrasts. We introduce the repeated measures proportional‐odds logistic regression model and describe for long ordinal outcomes modifications to the generalized estimating equation methodology used for parameter estimation. We introduce data from a trial assessing two surgical interventions, briefly describe and re‐analyse these using the new model and compare inferences from the new analysis with previously published results for the primary outcome measure (hip function at 12 months postoperatively). We use a simulation study to illustrate how this model also has more general application for conventional short ordinal scores, to select amongst competing models of varying complexity for the cut‐point parameters. Copyright © 2013 John Wiley & Sons, Ltd.     

Ordered multinomial regression for genetic association analysis of ordinal phenotypes at Biobank scale

Logistic regression is the primary analysis tool for binary traits in genome-wide association studies (GWAS). Multinomial regression extends logistic regression to multiple categories. However, many phenotypes more naturally take ordered, discrete values. Examples include (a) subtypes defined from multiple sources of clinical information and (b) derived phenotypes generated by specific phenotyping algorithms for electronic health records (EHR). GWAS of ordinal traits have been problematic. Dichotomizing can lead to a range of arbitrary cutoff values, generating inconsistent, hard to interpret results. Using multinomial regression ignores trait value hierarchy and potentially loses power. Treating ordinal data as quantitative can lead to misleading inference. To address these issues, we analyze ordinal traits with an ordered, multinomial model. This approach increases power and leads to more interpretable results. We derive efficient algorithms for computing test statistics, making ordinal trait GWAS computationally practical for Biobank scale data. Our method is available as a Julia package OrdinalGWAS.jl. Application to a COPDGene study confirms previously found signals based on binary case–control status, but with more significance. Additionally, we demonstrate the capability of our package to run on UK Biobank data by analyzing hypertension as an ordinal trait.     

Recommended Joint and Meta‐Analysis Strategies for Case‐Control Association Testing of Single Low‐Count Variants

In genome‐wide association studies of binary traits, investigators typically use logistic regression to test common variants for disease association within studies, and combine association results across studies using meta‐analysis. For common variants, logistic regression tests are well calibrated, and meta‐analysis of study‐specific association results is only slightly less powerful than joint analysis of the combined individual‐level data. In recent sequencing and dense chip based association studies, investigators increasingly test low‐frequency variants for disease association. In this paper, we seek to (1) identify the association test with maximal power among tests with well controlled type I error rate and (2) compare the relative power of joint and meta‐analysis tests. We use analytic calculation and simulation to compare the empirical type I error rate and power of four logistic regression based tests: Wald, score, likelihood ratio, and Firth bias‐corrected. We demonstrate for low‐count variants (roughly minor allele count [MAC] < 400) that: (1) for joint analysis, the Firth test has the best combination of type I error and power; (2) for meta‐analysis of balanced studies (equal numbers of cases and controls), the score test is best, but is less powerful than Firth test based joint analysis; and (3) for meta‐analysis of sufficiently unbalanced studies, all four tests can be anti‐conservative, particularly the score test. We also establish MAC as the key parameter determining test calibration for joint and meta‐analysis.     

Standardizing the power of the Hosmer–Lemeshow goodness of fit test in large data sets

The Hosmer–Lemeshow test is a commonly used procedure for assessing goodness of fit in logistic regression. It has, for example, been widely used for evaluation of risk‐scoring models. As with any statistical test, the power increases with sample size; this can be undesirable for goodness of fit tests because in very large data sets, small departures from the proposed model will be considered significant. By considering the dependence of power on the number of groups used in the Hosmer–Lemeshow test, we show how the power may be standardized across different sample sizes in a wide range of models. We provide and confirm mathematical derivations through simulation and analysis of data on 31,713 children from the Collaborative Perinatal Project. We make recommendations on how to choose the number of groups in the Hosmer–Lemeshow test based on sample size and provide example applications of the recommendations. Copyright © 2012 John Wiley & Sons, Ltd.     

On assessing model fit for distribution‐free longitudinal models under missing data

The generalized estimating equation (GEE), a distribution‐free, or semi‐parametric, approach for modeling longitudinal data, is used in a wide range of behavioral, psychotherapy, pharmaceutical drug safety, and healthcare‐related research studies. Most popular methods for assessing model fit are based on the likelihood function for parametric models, rendering them inappropriate for distribution‐free GEE. One rare exception is a score statistic initially proposed by Tsiatis for logistic regression (1980) and later extended by Barnhart and Willamson to GEE (1998). Because GEE only provides valid inference under the missing completely at random assumption and missing values arising in most longitudinal studies do not follow such a restricted mechanism, this GEE‐based score test has very limited applications in practice. We propose extensions of this goodness‐of‐fit test to address missing data under the missing at random assumption, a more realistic model that applies to most studies in practice. We examine the performance of the proposed tests using simulated data and demonstrate the utilities of such tests with data from a real study on geriatric depression and associated medical comorbidities. Copyright © 2013 John Wiley & Sons, Ltd.     

Development of a clinical prediction model for an ordinal outcome: the World Health Organization Multicentre Study of Clinical Signs and Etiological Agents of Pneumonia,Sepsis and Meningitis in Young Infants

This paper describes the methodologies used to develop a prediction model to assist health workers in developing countries in facing one of the most difficult health problems in all parts of the world: the presentation of an acutely ill young infant. Statistical approaches for developing the clinical prediction model faced at least two major difficulties. First, the number of predictor variables, especially clinical signs and symptoms, is very large, necessitating the use of data reduction techniques that are blinded to the outcome. Second, there is no uniquely accepted continuous outcome measure or final binary diagnostic criterion. For example, the diagnosis of neonatal sepsis is ill-defined. Clinical decision makers must identify infants likely to have positive cultures as well as to grade the severity of illness. In the WHO/ARI Young Infant Multicentre Study we have found an ordinal outcome scale made up of a mixture of laboratory and diagnostic markers to have several clinical advantages as well as to increase the power of tests for risk factors. Such a mixed ordinal scale does present statistical challenges because it may violate constant slope assumptions of ordinal regression models. In this paper we develop and validate an ordinal predictive model after choosing a data reduction technique. We show how ordinality of the outcome is checked against each predictor. We describe new but simple techniques for graphically examining residuals from ordinal logistic models to detect problems with variable transformations as well as to detect non-proportional odds and other lack of fit. We examine an alternative type of ordinal logistic model, the continuation ratio model, to determine if it provides a better fit. We find that it does not but that this model is easily modified to allow the regression coefficients to vary with cut-offs of the response variable. Complex terms in this extended model are penalized to allow only as much complexity as the data will support. We approximate the extended continuation ratio model with a model with fewer terms to allow us to draw a nomogram for obtaining various predictions. The model is validated for calibration and discrimination using the bootstrap. We apply much of the modelling strategy described in Harrell, Lee and Mark (Statist. Med. 15 , 361–387 (1998)) for survival analysis, adapting it to ordinal logistic regression and further emphasizing penalized maximum likelihood estimation and data reduction. © 1998 John Wiley & Sons, Ltd.     

Numeric score‐based conditional and overall change‐in‐status indices for ordered categorical data

Planned interventions and/or natural conditions often effect change on an ordinal categorical outcome (e.g., symptom severity). In such scenarios, it is sometimes desirable to assign a priori scores to observed changes in status, typically giving higher weight to changes of greater magnitude. We define change indices for such data based upon a multinomial model for each row of a c × c table, where the rows represent the baseline status categories. We distinguish an index designed to assess conditional changes within each baseline category from two others designed to capture overall change. One of these overall indices measures expected change across a target population. The other is scaled to capture the proportion of total possible change in the direction indicated by the data, so that it ranges from ?1 (when all subjects finish in the least favorable category) to +1 (when all finish in the most favorable category). The conditional assessment of change can be informative regardless of how subjects are sampled into the baseline categories. In contrast, the overall indices become relevant when subjects are randomly sampled at baseline from the target population of interest, or when the investigator is able to make certain assumptions about the baseline status distribution in that population. We use a Dirichlet‐multinomial model to obtain Bayesian credible intervals for the conditional change index that exhibit favorable small‐sample frequentist properties. Simulation studies illustrate the methods, and we apply them to examples involving changes in ordinal responses for studies of sleep deprivation and activities of daily living. Copyright © 2015 John Wiley & Sons, Ltd.     

Comparison of hypertabastic survival model with other unimodal hazard rate functions using a goodness‐of‐fit test

We studied the problem of testing a hypothesized distribution in survival regression models when the data is right censored and survival times are influenced by covariates. A modified chi‐squared type test, known as Nikulin‐Rao‐Robson statistic, is applied for the comparison of accelerated failure time models. This statistic is used to test the goodness‐of‐fit for hypertabastic survival model and four other unimodal hazard rate functions. The results of simulation study showed that the hypertabastic distribution can be used as an alternative to log‐logistic and log‐normal distribution. In statistical modeling, because of its flexible shape of hazard functions, this distribution can also be used as a competitor of Birnbaum‐Saunders and inverse Gaussian distributions. The results for the real data application are shown. Copyright © 2017 John Wiley & Sons, Ltd.     

Single‐marker and two‐marker association tests for unphased case‐control genotype data,with a power comparison

In case‐control single nucleotide polymorphism (SNP) data, the allele frequency, Hardy Weinberg Disequilibrium, and linkage disequilibrium (LD) contrast tests are three distinct sources of information about genetic association. While all three tests are typically developed in a retrospective context, we show that prospective logistic regression models may be developed that correspond conceptually to the retrospective tests. This approach provides a flexible framework for conducting a systematic series of association analyses using unphased genotype data and any number of covariates. For a single stage study, two single‐marker tests and four two‐marker tests are discussed. The true association models are derived and they allow us to understand why a model with only a linear term will generally fit well for a SNP in weak LD with a causal SNP, whatever the disease model, but not for a SNP in high LD with a non‐additive disease SNP. We investigate the power of the association tests using real LD parameters from chromosome 11 in the HapMap CEU population data. Among the single‐marker tests, the allelic test has on average the most power in the case of an additive disease, but for dominant, recessive, and heterozygote disadvantage diseases, the genotypic test has the most power. Among the four two‐marker tests, the Allelic‐LD contrast test, which incorporates linear terms for two markers and their interaction term, provides the most reliable power overall for the cases studied. Therefore, our result supports incorporating an interaction term as well as linear terms in multi‐marker tests. Genet. Epidemiol. 34:67–77, 2010. © 2009 Wiley‐Liss, Inc.     

Tests of calibration and goodness‐of‐fit in the survival setting

To access the calibration of a predictive model in a survival analysis setting, several authors have extended the Hosmer–Lemeshow goodness‐of‐fit test to survival data. Grønnesby and Borgan developed a test under the proportional hazards assumption, and Nam and D'Agostino developed a nonparametric test that is applicable in a more general survival setting for data with limited censoring. We analyze the performance of the two tests and show that the Grønnesby–Borgan test attains appropriate size in a variety of settings, whereas the Nam‐D'Agostino method has a higher than nominal Type 1 error when there is more than trivial censoring. Both tests are sensitive to small cell sizes. We develop a modification of the Nam‐D'Agostino test to allow for higher censoring rates. We show that this modified Nam‐D'Agostino test has appropriate control of Type 1 error and comparable power to the Grønnesby–Borgan test and is applicable to settings other than proportional hazards. We also discuss the application to small cell sizes. Copyright © 2015 John Wiley & Sons, Ltd.     

Testing goodness of fit in regression: a general approach for specified alternatives

When fitting generalized linear models or the Cox proportional hazards model, it is important to have tools to test for lack of fit. Because lack of fit comes in all shapes and sizes, distinguishing among different types of lack of fit is of practical importance. We argue that an adequate diagnosis of lack of fit requires a specified alternative model. Such specification identifies the type of lack of fit the test is directed against so that if we reject the null hypothesis, we know the direction of the departure from the model. The goodness‐of‐fit approach of this paper allows to treat different types of lack of fit within a unified general framework and to consider many existing tests as special cases. Connections with penalized likelihood and random effects are discussed, and the application of the proposed approach is illustrated with medical examples. Tailored functions for goodness‐of‐fit testing have been implemented in the R package globaltest . Copyright © 2012 John Wiley & Sons, Ltd.     

Global goodness‐of‐fit tests for group testing regression models

In a variety of biomedical applications, particularly those involving screening for infectious diseases, testing individuals (e.g. blood/urine samples, etc.) in pools has become a standard method of data collection. This experimental design, known as group testing (or pooled testing), can provide a large reduction in testing costs and can offer nearly the same precision as individual testing. To account for covariate information on individual subjects, regression models for group testing data have been proposed recently. However, there are currently no tools available to check the adequacy of these models. In this paper, we present various global goodness‐of‐fit tests for regression models with group testing data. We use simulation to examine the small‐sample size and power properties of the tests for different pool composition strategies. We illustrate our methods using two infectious disease data sets, one from an HIV study in Kenya and one from the Infertility Prevention Project. Copyright © 2009 John Wiley & Sons, Ltd.     

Prediction of ordinal outcomes when the association between predictors and outcome differs between outcome levels

There are a number of regression models which are widely used to predict ordinal outcomes. The commonly used models assume that all predictor variables have a similar effect at all levels of the outcome variable. If this is not the case, for example if some variables predict susceptibility to a disease and others predict the severity of the disease, then a more complex model is required. One possibility is the multinomial logistic regression model, which assumes that the predictor variables have different effects at all levels of the outcome variable. An alternative is to use the stereotype family of regression models. A one-dimensional stereotype model makes the assumption that the effect of each predictor is the same at all outcome levels. However, it is possible to fit stereotype models with more than one dimension, up to a maximum of min(k-1, p) where k is the number of outcome categories and p is the number of predictor variables. A stereotype model of this maximum dimension is equivalent to a multinomial logistic regression model, in that it will produce the same predicted values and log-likelihood. If there are sufficient outcome levels and/or predictor variables, there may be a number of stereotype models of differing dimension.The method is illustrated with an example of prediction of damage to joints in rheumatoid arthritis.     

Logistic‐AFT location‐scale mixture regression models with nonsusceptibility for left‐truncated and general interval‐censored data

In conventional survival analysis there is an underlying assumption that all study subjects are susceptible to the event. In general, this assumption does not adequately hold when investigating the time to an event other than death. Owing to genetic and/or environmental etiology, study subjects may not be susceptible to the disease. Analyzing nonsusceptibility has become an important topic in biomedical, epidemiological, and sociological research, with recent statistical studies proposing several mixture models for right‐censored data in regression analysis. In longitudinal studies, we often encounter left, interval, and right‐censored data because of incomplete observations of the time endpoint, as well as possibly left‐truncated data arising from the dissimilar entry ages of recruited healthy subjects. To analyze these kinds of incomplete data while accounting for nonsusceptibility and possible crossing hazards in the framework of mixture regression models, we utilize a logistic regression model to specify the probability of susceptibility, and a generalized gamma distribution, or a log‐logistic distribution, in the accelerated failure time location‐scale regression model to formulate the time to the event. Relative times of the conditional event time distribution for susceptible subjects are extended in the accelerated failure time location‐scale submodel. We also construct graphical goodness‐of‐fit procedures on the basis of the Turnbull–Frydman estimator and newly proposed residuals. Simulation studies were conducted to demonstrate the validity of the proposed estimation procedure. The mixture regression models are illustrated with alcohol abuse data from the Taiwan Aboriginal Study Project and hypertriglyceridemia data from the Cardiovascular Disease Risk Factor Two‐township Study in Taiwan. Copyright © 2013 John Wiley & Sons, Ltd.     

Propensity score‐based diagnostics for categorical response regression models

For binary or categorical response models, most goodness‐of‐fit statistics are based on the notion of partitioning the subjects into groups or regions and comparing the observed and predicted responses in these regions by a suitable chi‐squared distribution. Existing strategies create this partition based on the predicted response probabilities, or propensity scores, from the fitted model. In this paper, we follow a retrospective approach, borrowing the notion of balancing scores used in causal inference to inspect the conditional distribution of the predictors, given the propensity scores, in each category of the response to assess model adequacy. We can use this diagnostic under both prospective and retrospective sampling designs, and it may ascertain general forms of misspecification. We first present simple graphical and numerical summaries that can be used in a binary logistic model. We then generalize the tools to propose model diagnostics for the proportional odds model. We illustrate the methods with simulation studies and two data examples: (i) a case‐control study of the association between cumulative lead exposure and Parkinson's disease in the Boston, Massachusetts, area and (ii) and a cohort study of biomarkers possibly associated with diabetes, from the VA Normative Aging Study. Copyright © 2013 John Wiley & Sons, Ltd.     

A powerful approach to sub‐phenotype analysis in population‐based genetic association studies

The ultimate goal of genome‐wide association (GWA) studies is to identify genetic variants contributing effects to complex phenotypes in order to improve our understanding of the biological architecture underlying the trait. One approach to allow us to meet this challenge is to consider more refined sub‐phenotypes of disease, defined by pattern of symptoms, for example, which may be physiologically distinct, and thus may have different underlying genetic causes. The disadvantage of sub‐phenotype analysis is that large disease cohorts are sub‐divided into smaller case categories, thus reducing power to detect association. To address this issue, we have developed a novel test of association within a multinomial regression modeling framework, allowing for heterogeneity of genetic effects between sub‐phenotypes. The modeling framework is extremely flexible, and can be generalized to any number of distinct sub‐phenotypes. Simulations demonstrate the power of the multinomial regression‐based analysis over existing methods when genetic effects differ between sub‐phenotypes, with minimal loss of power when these effects are homogenous for the unified phenotype. Application of the multinomial regression analysis to a genome‐wide association study of type 2 diabetes, with cases categorized according to body mass index, highlights previously recognized differential mechanisms underlying obese and non‐obese forms of the disease, and provides evidence of a potential novel association that warrants follow‐up in independent replication cohorts. Genet. Epidemiol. 34: 335–343, 2010. © 2009 Wiley‐Liss, Inc.     

A Cochran–Armitage‐type and a score‐free global test for multivariate ordinal data

We propose a Cochran–Armitage‐type and a score‐free global test that can be used to assess the presence of an association between a set of ordinally scaled covariates and an outcome variable within the range of generalized linear models. Both tests are developed within the framework of the well‐established ‘global test’ methodology and as such are feasible in high‐dimensional data situations under any correlation and enable adjustment for covariates. The Cochran–Armitage‐type test, for which an intimate connection with the traditional score‐based Cochran–Armitage test is shown, rests upon explicit assumptions on the distances between the covariates' ordered categories. The score‐free test, in contrast, parametrizes these distances and thus keeps them flexible, rendering it ideally suited for covariates measured on an ordinal scale. As confirmed by means of simulations, the Cochran–Armitage‐type test focuses its power on set‐outcome relationships where the distances between the covariates' categories are equal or close to those assumed, whereas the score‐free test spreads its power over a wide range of possible set‐outcome relationships, putting more emphasis on monotonic than on non‐monotonic ones. Based on the tests' power properties, it is discussed when to favour one or the other, and the practical merits of both of them are illustrated by an application in the field of rehabilitation medicine. Our proposed tests are implemented in the R package globaltest . Copyright © 2016 John Wiley & Sons, Ltd.