Evaluating markers for treatment selection based on survival time

For many medical conditions, several treatment options may be available for treating patients. We consider evaluating markers based on a simple treatment selection policy that incorporates information on the patient's marker value. For example, colon cancer patients may be treated by surgery alone or surgery plus chemotherapy. The c-myc gene expression level may be used as a biomarker for treatment selection. Although traditional regression methods may assess the effect of the marker and treatment on outcomes, it is more appealing to quantify directly the potential impact on the population of using the marker to select treatment. A useful tool is the selection impact (SI) curve proposed by Song and Pepe for binary outcomes (Biometrics 2004; 60:874-883). However, the current SI method does not deal with continuous outcomes, nor does it allow to adjust for other covariates that are important for treatment selection. In this paper, we extend the SI curve for general outcomes, with a specific focus on survival time. We further propose the covariate-specific SI curve to incorporate covariate information in treatment selection. Nonparametric and semiparametric estimators are developed accordingly. We show that the proposed estimators are consistent and asymptotically normal. The performance is assessed by simulation studies and illustrated through an application to data from a cancer clinical trial.     

A model‐free estimation for the covariate‐adjusted Youden index and its associated cut‐point

In medical research, continuous markers are widely employed in diagnostic tests to distinguish diseased and non‐diseased subjects. The accuracy of such diagnostic tests is commonly assessed using the receiver operating characteristic (ROC) curve. To summarize an ROC curve and determine its optimal cut‐point, the Youden index is popularly used. In literature, the estimation of the Youden index has been widely studied via various statistical modeling strategies on the conditional density. This paper proposes a new model‐free estimation method, which directly estimates the covariate‐adjusted cut‐point without estimating the conditional density. Consequently, covariate‐adjusted Youden index can be estimated based on the estimated cut‐point. The proposed method formulates the estimation problem in a large margin classification framework, which allows flexible modeling of the covariate‐adjusted Youden index through kernel machines. The advantage of the proposed method is demonstrated in a variety of simulated experiments as well as a real application to Pima Indians diabetes study. Copyright © 2014 John Wiley & Sons, Ltd.     

A covariate-specific time-dependent receiver operating characteristic curve for correlated survival data

Several studies for the clinical validity of circulating tumor cells (CTCs) in metastatic breast cancer were conducted showing that it is a prognostic biomarker of overall survival. In this work, we consider an individual patient data meta-analysis for nonmetastatic breast cancer to assess the discrimination of CTCs regarding the risk of death. Data are collected in several centers and present correlated failure times for subjects of the same center. However, although the covariate-specific time-dependent receiver operating characteristic (ROC) curve has been widely used for assessing the performance of a biomarker, there is no methodology yet that can handle this specific setting with clustered censored failure times. We propose an estimator for the covariate-specific time-dependent ROC curves and area under the ROC curve when clustered failure times are detected. We discuss the assumptions under which the estimators are consistent and their interpretations. We assume a shared frailty model for modeling the effect of the covariates and the biomarker on the outcome in order to account for the cluster effect. A simulation study was conducted and it shows negligible bias for the proposed estimator and a nonparametric one based on inverse probability censoring weighting, while a semiparametric estimator, ignoring the clustering, is markedly biased. Finally, in our application to breast cancer data, the estimation of the covariate-specific area under the curves illustrates that the CTCs discriminate better patients with inflammatory tumor than patients with noninflammatory tumor, with respect to their risk of death.     

Adjusting the generalized ROC curve for covariates

Receiver operating characteristic (ROC) curves and in particular the area under the curve (AUC), are widely used to examine the effectiveness of diagnostic markers. Diagnostic markers and their corresponding ROC curves can be strongly influenced by covariate variables. When several diagnostic markers are available, they can be combined by a best linear combination such that the area under the ROC curve of the combination is maximized among all possible linear combinations. In this paper we discuss covariate effects on this linear combination assuming that the multiple markers, possibly transformed, follow a multivariate normal distribution. The ROC curve of this linear combination when markers are adjusted for covariates is estimated and approximate confidence intervals for the corresponding AUC are derived. An example of two biomarkers of coronary heart disease for which covariate information on age and gender is available is used to illustrate this methodology.     

Propensity score estimation with missing values using a multiple imputation missingness pattern (MIMP) approach

Propensity scores have been used widely as a bias reduction method to estimate the treatment effect in nonrandomized studies. Since many covariates are generally included in the model for estimating the propensity scores, the proportion of subjects with at least one missing covariate could be large. While many methods have been proposed for propensity score‐based estimation in the presence of missing covariates, little has been published comparing the performance of these methods. In this article we propose a novel method called multiple imputation missingness pattern (MIMP) and compare it with the naive estimator (ignoring propensity score) and three commonly used methods of handling missing covariates in propensity score‐based estimation (separate estimation of propensity scores within each pattern of missing data, multiple imputation and discarding missing data) under different mechanisms of missing data and degree of correlation among covariates. Simulation shows that all adjusted estimators are much less biased than the naive estimator. Under certain conditions MIMP provides benefits (smaller bias and mean‐squared error) compared with existing alternatives. Copyright © 2009 John Wiley & Sons, Ltd.     

Evaluating classification performance of biomarkers in two-phase case-control studies

Biomarkers are playing an increasingly important role in disease screening, early detection, and risk prediction. The two-phase case-control sampling study design is widely used for the evaluation of candidate biomarkers. The sampling probabilities for cases and controls in the second phase can often depend on other covariates (sampling strata). This biased sampling can lead to invalid inference on a biomarker's classification accuracy if not properly accounted for. In this paper, we adopt the idea of inverse probability weighting and develop inverse probability weighting–based estimators for various measures of a biomarker's classification performance, including the points on the receiver operating characteristics (ROCs) curve, the area under the ROC curve (area under the curve), and the partial area under the curve. In particular, we consider classification accuracy estimators using sampling weights estimated conditionally on sampling strata and further improve their efficiency through the use of estimated weights that additionally take into account the auxiliary variables available from the phase-one cohort. We develop asymptotic properties of the proposed estimators and provide analytical variance for making inference. Extensive simulation studies demonstrate excellent performance of the proposed weighted estimators, while the traditional empirical estimator can be severely biased. We also investigate the advantages in efficiency gain for estimating various classification accuracy estimators through the use of auxiliary variables in addition to sampling strata and apply the proposed method to examples from a renal artery stenosis study and a prostate cancer study.     

Estimating restricted mean treatment effects with stacked survival models

The difference in restricted mean survival times between two groups is a clinically relevant summary measure. With observational data, there may be imbalances in confounding variables between the two groups. One approach to account for such imbalances is estimating a covariate‐adjusted restricted mean difference by modeling the covariate‐adjusted survival distribution and then marginalizing over the covariate distribution. Because the estimator for the restricted mean difference is defined by the estimator for the covariate‐adjusted survival distribution, it is natural to expect that a better estimator of the covariate‐adjusted survival distribution is associated with a better estimator of the restricted mean difference. We therefore propose estimating restricted mean differences with stacked survival models. Stacked survival models estimate a weighted average of several survival models by minimizing predicted error. By including a range of parametric, semi‐parametric, and non‐parametric models, stacked survival models can robustly estimate a covariate‐adjusted survival distribution and, therefore, the restricted mean treatment effect in a wide range of scenarios. We demonstrate through a simulation study that better performance of the covariate‐adjusted survival distribution often leads to better mean squared error of the restricted mean difference although there are notable exceptions. In addition, we demonstrate that the proposed estimator can perform nearly as well as Cox regression when the proportional hazards assumption is satisfied and significantly better when proportional hazards is violated. Finally, the proposed estimator is illustrated with data from the United Network for Organ Sharing to evaluate post‐lung transplant survival between large‐volume and small‐volume centers. Copyright © 2016 John Wiley & Sons, Ltd.     

Covariate adjustment in randomized trials with binary outcomes: targeted maximum likelihood estimation

Covariate adjustment using linear models for continuous outcomes in randomized trials has been shown to increase efficiency and power over the unadjusted method in estimating the marginal effect of treatment. However, for binary outcomes, investigators generally rely on the unadjusted estimate as the literature indicates that covariate-adjusted estimates based on the logistic regression models are less efficient. The crucial step that has been missing when adjusting for covariates is that one must integrate/average the adjusted estimate over those covariates in order to obtain the marginal effect. We apply the method of targeted maximum likelihood estimation (tMLE) to obtain estimators for the marginal effect using covariate adjustment for binary outcomes. We show that the covariate adjustment in randomized trials using the logistic regression models can be mapped, by averaging over the covariate(s), to obtain a fully robust and efficient estimator of the marginal effect, which equals a targeted maximum likelihood estimator. This tMLE is obtained by simply adding a clever covariate to a fixed initial regression. We present simulation studies that demonstrate that this tMLE increases efficiency and power over the unadjusted method, particularly for smaller sample sizes, even when the regression model is mis-specified.     

Point estimation following two‐stage adaptive threshold enrichment clinical trials

Recently, several study designs incorporating treatment effect assessment in biomarker‐based subpopulations have been proposed. Most statistical methodologies for such designs focus on the control of type I error rate and power. In this paper, we have developed point estimators for clinical trials that use the two‐stage adaptive enrichment threshold design. The design consists of two stages, where in stage 1, patients are recruited in the full population. Stage 1 outcome data are then used to perform interim analysis to decide whether the trial continues to stage 2 with the full population or a subpopulation. The subpopulation is defined based on one of the candidate threshold values of a numerical predictive biomarker. To estimate treatment effect in the selected subpopulation, we have derived unbiased estimators, shrinkage estimators, and estimators that estimate bias and subtract it from the naive estimate. We have recommended one of the unbiased estimators. However, since none of the estimators dominated in all simulation scenarios based on both bias and mean squared error, an alternative strategy would be to use a hybrid estimator where the estimator used depends on the subpopulation selected. This would require a simulation study of plausible scenarios before the trial.     

Maximizing an ROC‐type measure via linear combination of markers when the gold reference is continuous

Effectively combining many classification instruments or diagnostic measurements together to improve the classification accuracy of individuals is a common idea in disease diagnosis or classification. These ensemble‐type diagnostic methods can be constructed with respect to different kinds of performance criterions. Among them, the receiver operating characteristic (ROC) curve is the most popular criterion, which, together with some indexes derived from it, is commonly used to evaluate and summarize the performance of a classification instrument, such as a biomarker or a classifier. However, the usefulness of ROC curve and its related indexes relies on the existence of a binary label for each individual subject. In many disease diagnosis situations, such a binary variable may not exist, but only the continuous measurement of the true disease status is available. This true disease status is often referred to as the ‘gold standard’. The modified area under ROC curve (AUC)‐type measure defined by Obuchowski is a method proposed to accommodate such a situation. However, there is still no method for finding the optimal combination of diagnostic measurements, with respect to such an index, to have better diagnostic power than that of each individual measurement. In this paper, we propose an algorithm for finding the optimal combination with respect to such an extended AUC‐type measure such that the combined measurement can have more diagnostic power. We illustrate the performance of our algorithm by using some synthesized data and a diabetes data set. Copyright © 2012 John Wiley & Sons, Ltd.     

Variance reduction in randomised trials by inverse probability weighting using the propensity score

In individually randomised controlled trials, adjustment for baseline characteristics is often undertaken to increase precision of the treatment effect estimate. This is usually performed using covariate adjustment in outcome regression models. An alternative method of adjustment is to use inverse probability‐of‐treatment weighting (IPTW), on the basis of estimated propensity scores. We calculate the large‐sample marginal variance of IPTW estimators of the mean difference for continuous outcomes, and risk difference, risk ratio or odds ratio for binary outcomes. We show that IPTW adjustment always increases the precision of the treatment effect estimate. For continuous outcomes, we demonstrate that the IPTW estimator has the same large‐sample marginal variance as the standard analysis of covariance estimator. However, ignoring the estimation of the propensity score in the calculation of the variance leads to the erroneous conclusion that the IPTW treatment effect estimator has the same variance as an unadjusted estimator; thus, it is important to use a variance estimator that correctly takes into account the estimation of the propensity score. The IPTW approach has particular advantages when estimating risk differences or risk ratios. In this case, non‐convergence of covariate‐adjusted outcome regression models frequently occurs. Such problems can be circumvented by using the IPTW adjustment approach. © 2013 The authors. Statistics in Medicine published by John Wiley & Sons, Ltd.     

Smooth semiparametric receiver operating characteristic curves for continuous diagnostic tests

We propose a semiparametric kernel distribution function estimator, based on which a new smooth semiparametric estimator of the receiver operating characteristic (ROC) curve is constructed. We derive the asymptotic bias and variance of the newly proposed distribution function estimator and show that it is more efficient than the traditional non-parametric kernel distribution estimator. We also derive the asymptotic bias and variance of our new ROC curve estimator and show that it is more efficient than the smooth non-parametric ROC curve estimator proposed by Zou et al. (Stat. Med. 1997; 16:2143-2156) and Lloyd (J. Am. Stat. Assoc. 1998; 93:1356-1364). For our proposed estimators, we derive data-based methods for bandwidth selection. In addition, we present some results on the analysis of two real data sets. Finally, a simulation study is presented to show that our estimators are better than the non-parametric counterparts in terms of bias, standard error, and mean-square error.     

Quantifying the bias due to observed individual confounders in causal treatment effect estimates

It is often of interest to use observational data to estimate the causal effect of a target exposure or treatment on an outcome. When estimating the treatment effect, it is essential to appropriately adjust for selection bias due to observed confounders using, for example, propensity score weighting. Selection bias due to confounders occurs when individuals who are treated are substantially different from those who are untreated with respect to covariates that are also associated with the outcome. A comparison of the unadjusted, naive treatment effect estimate with the propensity score adjusted treatment effect estimate provides an estimate of the selection bias due to these observed confounders. In this article, we propose methods to identify the observed covariate that explains the largest proportion of the estimated selection bias. Identification of the most influential observed covariate or covariates is important in resource-sensitive settings where the number of covariates obtained from individuals needs to be minimized due to cost and/or patient burden and in settings where this covariate can provide actionable information to healthcare agencies, providers, and stakeholders. We propose straightforward parametric and nonparametric procedures to examine the role of observed covariates and quantify the proportion of the observed selection bias explained by each covariate. We demonstrate good finite sample performance of our proposed estimates using a simulation study and use our procedures to identify the most influential covariates that explain the observed selection bias in estimating the causal effect of alcohol use on progression of Huntington's disease, a rare neurological disease.     

Easy and accurate variance estimation of the nonparametric estimator of the partial area under the ROC curve and its application

The receiver operating characteristic (ROC) curve is a popular technique with applications, for example, investigating an accuracy of a biomarker to delineate between disease and non‐disease groups. A common measure of accuracy of a given diagnostic marker is the area under the ROC curve (AUC). In contrast with the AUC, the partial area under the ROC curve (pAUC) looks into the area with certain specificities (i.e., true negative rate) only, and it can be often clinically more relevant than examining the entire ROC curve. The pAUC is commonly estimated based on a U‐statistic with the plug‐in sample quantile, making the estimator a non‐traditional U‐statistic. In this article, we propose an accurate and easy method to obtain the variance of the nonparametric pAUC estimator. The proposed method is easy to implement for both one biomarker test and the comparison of two correlated biomarkers because it simply adapts the existing variance estimator of U‐statistics. In this article, we show accuracy and other advantages of the proposed variance estimation method by broadly comparing it with previously existing methods. Further, we develop an empirical likelihood inference method based on the proposed variance estimator through a simple implementation. In an application, we demonstrate that, depending on the inferences by either the AUC or pAUC, we can make a different decision on a prognostic ability of a same set of biomarkers. Copyright © 2016 John Wiley & Sons, Ltd.     

Best linear inverse probability weighted estimation for two-phase designs and missing covariate regression

The inverse probability weighted estimator is often applied to two-phase designs and regression with missing covariates. Inverse probability weighted estimators typically are less efficient than likelihood-based estimators but, in general, are more robust against model misspecification. In this paper, we propose a best linear inverse probability weighted estimator for two-phase designs and missing covariate regression. Our proposed estimator is the projection of the SIPW onto the orthogonal complement of the score space based on a working regression model of the observed covariate data. The efficiency gain is from the use of the association between the outcome variable and the available covariates, which is the working regression model. One advantage of the proposed estimator is that there is no need to calculate the augmented term of the augmented weighted estimator. The estimator can be applied to general missing data problems or two-phase design studies in which the second phase data are obtained in a subcohort. The method can also be applied to secondary trait case-control genetic association studies. The asymptotic distribution is derived, and the finite sample performance of the proposed estimator is examined via extensive simulation studies. The methods are applied to a bladder cancer case-control study.     

Group sequential testing of the predictive accuracy of a continuous biomarker with unknown prevalence

Group sequential testing procedures have been proposed as an approach to conserving resources in biomarker validation studies. Previously, we derived the asymptotic properties of the sequential empirical positive predictive value (PPV) and negative predictive value (NPV) curves, which summarize the predictive accuracy of a continuous marker, under case‐control sampling. A limitation of this approach is that the prevalence cannot be estimated from a case‐control study and must be assumed known. In this paper, we consider group sequential testing of the predictive accuracy of a continuous biomarker with unknown prevalence. First, we develop asymptotic theory for the sequential empirical PPV and NPV curves when the prevalence must be estimated, rather than assumed known in a case‐control study. We then discuss how our results can be combined with standard group sequential methods to develop group sequential testing procedures and bias‐adjusted estimators for the PPV and NPV curve. The small sample properties of the proposed group sequential testing procedures and estimators are evaluated by simulation, and we illustrate our approach in the context of a study to validate a novel biomarker for prostate cancer. Copyright © 2015 John Wiley & Sons, Ltd.     

Recurrent event data analysis with intermittently observed time‐varying covariates

Although recurrent event data analysis is a rapidly evolving area of research, rigorous studies on estimation of the effects of intermittently observed time‐varying covariates on the risk of recurrent events have been lacking. Existing methods for analyzing recurrent event data usually require that the covariate processes are observed throughout the entire follow‐up period. However, covariates are often observed periodically rather than continuously. We propose a novel semiparametric estimator for the regression parameters in the popular proportional rate model. The proposed estimator is based on an estimated score function where we kernel smooth the mean covariate process. We show that the proposed semiparametric estimator is asymptotically unbiased, normally distributed, and derives the asymptotic variance. Simulation studies are conducted to compare the performance of the proposed estimator and the simple methods carrying forward the last covariates. The different methods are applied to an observational study designed to assess the effect of group A streptococcus on pharyngitis among school children in India. Copyright © 2016 John Wiley & Sons, Ltd.     

Conditional estimation of sensitivity and specificity from a phase 2 biomarker study allowing early termination for futility

Development of a disease screening biomarker involves several phases. In phase 2 its sensitivity and specificity is compared with established thresholds for minimally acceptable performance. Since we anticipate that most candidate markers will not prove to be useful and availability of specimens and funding is limited, early termination of a study is appropriate, if accumulating data indicate that the marker is inadequate. Yet, for markers that complete phase 2, we seek estimates of sensitivity and specificity to proceed with the design of subsequent phase 3 studies. We suggest early stopping criteria and estimation procedures that adjust for bias caused by the early termination option. An important aspect of our approach is to focus on properties of estimates conditional on reaching full study enrollment. We propose the conditional‐UMVUE and contrast it with other estimates, including naïve estimators, the well‐studied unconditional‐UMVUE and the mean and median Whitehead‐adjusted estimators. The conditional‐UMVUE appears to be a very good choice. Copyright © 2008 John Wiley & Sons, Ltd.     

Corrected score estimation in the proportional hazards model with misclassified discrete covariates

We consider Cox proportional hazards regression when the covariate vector includes error-prone discrete covariates along with error-free covariates, which may be discrete or continuous. The misclassification in the discrete error-prone covariates is allowed to be of any specified form. Building on the work of Nakamura and his colleagues, we present a corrected score method for this setting. The method can handle all three major study designs (internal validation design, external validation design, and replicate measures design), both functional and structural error models, and time-dependent covariates satisfying a certain 'localized error' condition. We derive the asymptotic properties of the method and indicate how to adjust the covariance matrix of the regression coefficient estimates to account for estimation of the misclassification matrix. We present the results of a finite-sample simulation study under Weibull survival with a single binary covariate having known misclassification rates. The performance of the method described here was similar to that of related methods we have examined in previous works. Specifically, our new estimator performed as well as or, in a few cases, better than the full Weibull maximum likelihood estimator. We also present simulation results for our method for the case where the misclassification probabilities are estimated from an external replicate measures study. Our method generally performed well in these simulations. The new estimator has a broader range of applicability than many other estimators proposed in the literature, including those described in our own earlier work, in that it can handle time-dependent covariates with an arbitrary misclassification structure. We illustrate the method on data from a study of the relationship between dietary calcium intake and distal colon cancer.