Validation of surrogate endpoints in cancer clinical trials via principal stratification with an application to a prostate cancer trial

Increasing attention has been focused on the use and validation of surrogate endpoints in cancer clinical trials. Previous literature on validation of surrogate endpoints are classified into four approaches: the proportion explained approach; the indirect effects approach; the meta‐analytic approach; and the principal stratification approach. The mainstream in cancer research has seen the application of a meta‐analytic approach. However, VanderWeele (2013) showed that all four of these approaches potentially suffer from the surrogate paradox. It was also shown that, if a principal surrogate satisfies additional criteria called one‐sided average causal sufficiency, the surrogate cannot exhibit a surrogate paradox. Here, we propose a method for estimating principal effects under a monotonicity assumption. Specifically, we consider cancer clinical trials which compare a binary surrogate endpoint and a time‐to‐event clinical endpoint under two naturally ordered treatments (e.g. combined therapy vs. monotherapy). Estimation based on a mean score estimating equation will be implemented by the expectation‐maximization algorithm. We will also apply the proposed method as well as other surrogacy criteria to evaluate the surrogacy of prostate‐specific antigen using data from a phase III advanced prostate cancer trial, clarifying the complementary roles of both the principal stratification and meta‐analytic approaches in the evaluation of surrogate endpoints in cancer. Copyright © 2017 John Wiley & Sons, Ltd.     

A rank test for bivariate time‐to‐event outcomes when one event is a surrogate

In many clinical settings, improving patient survival is of interest but a practical surrogate, such as time to disease progression, is instead used as a clinical trial's primary endpoint. A time‐to‐first endpoint (e.g., death or disease progression) is commonly analyzed but may not be adequate to summarize patient outcomes if a subsequent event contains important additional information. We consider a surrogate outcome very generally as one correlated with the true endpoint of interest. Settings of interest include those where the surrogate indicates a beneficial outcome so that the usual time‐to‐first endpoint of death or surrogate event is nonsensical. We present a new two‐sample test for bivariate, interval‐censored time‐to‐event data, where one endpoint is a surrogate for the second, less frequently observed endpoint of true interest. This test examines whether patient groups have equal clinical severity. If the true endpoint rarely occurs, the proposed test acts like a weighted logrank test on the surrogate; if it occurs for most individuals, then our test acts like a weighted logrank test on the true endpoint. If the surrogate is a useful statistical surrogate, our test can have better power than tests based on the surrogate that naively handles the true endpoint. In settings where the surrogate is not valid (treatment affects the surrogate but not the true endpoint), our test incorporates the information regarding the lack of treatment effect from the observed true endpoints and hence is expected to have a dampened treatment effect compared with tests based on the surrogate alone. Published 2016. This article is a U.S. Government work and is in the public domain in the USA.     

Evaluation of surrogate endpoints in randomized experiments with mixed discrete and continuous outcomes

A statistical definition of surrogate endpoints as well as validation criteria was first presented by Prentice. Freedman et al. supplemented these criteria with the so-called proportion explained. Buyse and Molenberghs pointed to inadequacies of these criteria and suggested a new definition of surrogacy based on (i) the relative effect linking the overall effect of treatment on both endpoints and (ii) an individual-level measure of agreement between both endpoints. Using data from a randomized trial, they showed how a potential surrogate endpoint can be studied using a joint model for the surrogate and the true endpoint. Whereas Buyse and Molenberghs restricted themselves to the fairly simple cases of jointly normal and jointly binary outcomes, we treat the situation where the surrogate is binary and the true endpoint is continuous, or vice versa. In addition, we consider the case of ordinal endpoints. Further, Buyse et al. extended the approach of Buyse and Molenberghs to a meta-analytic context. We will adopt a similar approach for responses of a mixed data type.     

Five criteria for using a surrogate endpoint to predict treatment effect based on data from multiple previous trials

A surrogate endpoint in a randomized clinical trial is an endpoint that occurs after randomization and before the true, clinically meaningful, endpoint that yields conclusions about the effect of treatment on true endpoint. A surrogate endpoint can accelerate the evaluation of new treatments but at the risk of misleading conclusions. Therefore, criteria are needed for deciding whether to use a surrogate endpoint in a new trial. For the meta‐analytic setting of multiple previous trials, each with the same pair of surrogate and true endpoints, this article formulates 5 criteria for using a surrogate endpoint in a new trial to predict the effect of treatment on the true endpoint in the new trial. The first 2 criteria, which are easily computed from a zero‐intercept linear random effects model, involve statistical considerations: an acceptable sample size multiplier and an acceptable prediction separation score. The remaining 3 criteria involve clinical and biological considerations: similarity of biological mechanisms of treatments between the new trial and previous trials, similarity of secondary treatments following the surrogate endpoint between the new trial and previous trials, and a negligible risk of harmful side effects arising after the observation of the surrogate endpoint in the new trial. These 5 criteria constitute an appropriately high bar for using a surrogate endpoint to make a definitive treatment recommendation.     

A method to address between‐subject heterogeneity for identification of principal surrogate markers in repeated low‐dose challenge HIV vaccine studies

Repeated low‐dose challenge designs in nonhuman primate studies have recently received attention in the literature as a means of evaluating vaccines for HIV prevention and identifying immune surrogates for their protective effects. Existing methods for surrogate identification in this type of study design rely on the assumption of homogeneity across subjects (namely, independent infection risks after each challenge within each subject and conditional on covariates). In practice, random variation across subjects is likely to occur because of unmeasured biologic factors. Failure to account for this heterogeneity or within‐subject correlation can result in biased inference regarding the surrogate value of immune biomarkers and underpowered study designs for detecting surrogate endpoints. In this paper, we adopt a discrete‐time survival model with random effects to account for between‐subject heterogeneity, and we develop estimators and testing procedures for evaluating principal surrogacy of immune biomarkers. Simulation studies reveal that the heterogeneous model achieves substantial bias reduction compared to the homogeneous model, with little cost of efficiency. We recommend the use of this heterogeneous model as a complementary tool to existing methods when designing and analyzing repeated low‐dose challenge studies for evaluating surrogate endpoints.     

Comparing and combining biomarkers as principal surrogates for time‐to‐event clinical endpoints

Principal surrogate endpoints are useful as targets for phase I and II trials. In many recent trials, multiple post‐randomization biomarkers are measured. However, few statistical methods exist for comparison of or combination of biomarkers as principal surrogates, and none of these methods to our knowledge utilize time‐to‐event clinical endpoint information. We propose a Weibull model extension of the semi‐parametric estimated maximum likelihood method that allows for the inclusion of multiple biomarkers in the same risk model as multivariate candidate principal surrogates. We propose several methods for comparing candidate principal surrogates and evaluating multivariate principal surrogates. These include the time‐dependent and surrogate‐dependent true and false positive fraction, the time‐dependent and the integrated standardized total gain, and the cumulative distribution function of the risk difference. We illustrate the operating characteristics of our proposed methods in simulations and outline how these statistics can be used to evaluate and compare candidate principal surrogates. We use these methods to investigate candidate surrogates in the Diabetes Control and Complications Trial. Copyright © 2014 John Wiley & Sons, Ltd.     

A new proportion measure of the treatment effect captured by candidate surrogate endpoints

The use of surrogate endpoints is expected to play an important role in the development of new drugs, as they can be used to reduce the sample size and/or duration of randomized clinical trials. Biostatistical researchers and practitioners have proposed various surrogacy measures; however, (i) most of these surrogacy measures often fall outside the range [0,1] without any assumptions, (ii) these surrogacy measures do not provide a cut‐off value for judging a surrogacy level of candidate surrogate endpoints, and (iii) most surrogacy measures are highly variable; thus, the confidence intervals are often unacceptably wide. In order to solve problems (i) and (ii), we propose a new surrogacy measure, a proportion of the treatment effect captured by candidate surrogate endpoints (PCS), on the basis of the decomposition of the treatment effect into parts captured and non‐captured by the candidate surrogate endpoints. In order to solve problem (iii), we propose an estimation method based on the half‐range mode method with the bootstrap distribution of the estimated surrogacy measures. Finally, through numerical experiments and two empirical examples, we show that the PCS with the proposed estimation method overcomes these difficulties. The results of this paper contribute to the reliable evaluation of how much of the treatment effect is captured by candidate surrogate endpoints. Copyright © 2014 John Wiley & Sons, Ltd.     

Exploring the relationship between the causal‐inference and meta‐analytic paradigms for the evaluation of surrogate endpoints

Nowadays, two main frameworks for the evaluation of surrogate endpoints, based on causal‐inference and meta‐analysis, dominate the scene. Earlier work showed that the metrics of surrogacy introduced in both paradigms are related, although in a complex way that is difficult to study analytically. In the present work, this relationship is further examined using simulations and the analysis of a case study. The results indicate that the extent to which both paradigms lead to similar conclusions regarding the validity of the surrogate, depends on a complex interplay between multiple factors like the ratio of the between and within trial variability and the unidentifiable correlations between the potential outcomes. All the analyses were carried out using the newly developed R package Surrogate, which is freely available via CRAN. Copyright © 2015 John Wiley & Sons, Ltd.     

Does the Prentice criterion validate surrogate endpoints?

Randomized Phase II or Phase III clinical trials that are powered based on clinical endpoints, such as survival time, may be prohibitively expensive, in terms of both the time required for their completion and the number of patients required. As such, surrogate endpoints, such as objective tumour response or markers including prostate specific antigen or CA-125, have gained widespread popularity in clinical trials. If an improvement in a surrogate endpoint does not itself confer patient benefit, then consideration must be given to the extent to which improvement in a surrogate endpoint implies improvement in the true clinical endpoint of interest. That this is not a trivial issue is demonstrated by the results of an NIH-sponsored trial of anti-arrhythmic drugs, in which the ability to correct an irregular heart beat not only did not correspond to a survival benefit but in fact led to excess mortality. One approach to the validation of surrogate endpoints involves ensuring that a valid between-group analysis of the surrogate endpoint constitutes also a valid analysis of the true clinical endpoint. The Prentice criterion is a set of conditions that essentially specify the conditional independence of the impact of treatment on the true endpoint, given the surrogate endpoint. It is shown that this criterion alone ensures that an observed effect of the treatment on the true endpoint implies a treatment effect also on the surrogate endpoint, but contrary to popular belief, it does not ensure the converse, specifically that the observation of a significant treatment effect on the surrogate endpoint can be used to infer a treatment effect on the true endpoint.     

Surrogate and auxiliary endpoints in clinical trials,with potential applications in cancer and aids research

Surrogate endpoints have been defined by Prentice as response variables that can substitute for a ‘true’ endpoint for the purpose of comparing specific interventions or treatments in a clinical trial. The applicability of this definition, and of related surrogate endpoint criteria, is discussed, with emphasis on cancer and AIDS research settings. Auxiliary endpoints are defined as response variables, or covariates, that can strengthen true endpoint analyses. Specifically, such response variables provide some additional information on true endpoint occurrence times for study subjects having censored values for such times. Auxiliary variables will very frequently be available, and they may be able to be used without making additional strong assumptions. Approaches to the use of auxiliary variables using ideas based on augmented score and augmented likelihood methods are described.     

Estimating treatment difference for binary responses in the presence of surrogate endpoints

Surrogate endpoint is chosen as a measure or indicator of a biological process that is obtained sooner, at less cost or less invasively than a true endpoint of health outcome, and is used to arrive at conclusions about the effect of intervention on the true endpoint. Following the work of Begg and Leung (J. R. Stat. Soc. Ser. A 2000; 163:15-28), we introduce a new motivation to analyse the surrogate and true endpoints together, to have a better estimate of the treatment difference where both the endpoints are binary. Several estimators are studied and compared. Some real data sets are analysed.     

Nonparametric meta‐analysis for diagnostic accuracy studies

Summarizing the information of many studies using a meta‐analysis becomes more and more important, also in the field of diagnostic studies. The special challenge in meta‐analysis of diagnostic accuracy studies is that in general sensitivity and specificity are co‐primary endpoints. Across the studies both endpoints are correlated, and this correlation has to be considered in the analysis. The standard approach for such a meta‐analysis is the bivariate logistic random effects model. An alternative approach is to use marginal beta‐binomial distributions for the true positives and the true negatives, linked by copula distributions. In this article, we propose a new, nonparametric approach of analysis, which has greater flexibility with respect to the correlation structure, and always converges. In a simulation study, it becomes apparent that the empirical coverage of all three approaches is in general below the nominal level. Regarding bias, empirical coverage, and mean squared error the nonparametric model is often superior to the standard model, and comparable with the copula model. The three approaches are also applied to two example meta‐analyses. Copyright © 2015 John Wiley & Sons, Ltd.     

Surrogate endpoints in clinical trials: cancer

Investigators use a surrogate endpoint when the endpoint of interest is too difficult and/or expensive to measure routinely and when they can define some other, more readily measurable, endpoint, that is sufficiently well correlated with the first to justify its use as a substitute. A surrogate endpoint is usually proposed on the basis of a biologic rationale. In cancer studies with survival time as the primary endpoint, surrogate endpoints frequently employed are tumour response, time to progression, or time to reappearance of disease, since these events occur earlier and are unaffected by use of secondary therapies. In early drug development studies, tumour response is often the true primary endpoint. We discuss the investigation of the validity of carcinoembryonic antigen (a tumour marker present in the blood) as a surrogate for tumour response. In considering the validity of surrogate endpoints, one must distinguish between study endpoints that provide a basis for reliable comparisons of therapeutic effect, and clinical endpoints that are useful for patient management but have insufficient sensitivity and/or specificity to provide reproducible assessments of the effects of particular therapies.     

On the relationship between association and surrogacy when both the surrogate and true endpoint are binary outcomes

The relationship between association and surrogacy has been the focus of much debate in the surrogate marker literature. Recently, the individual causal association (ICA) has been introduced as a metric of surrogacy in the causal inference framework, when both the surrogate and the true endpoint are normally distributed and when both are binary. Earlier work on the normal case has demonstrated that, although the ICA and the adjusted association are related metrics, their relationship strongly depends on unidentifiable parameters and, consequently, the association between both endpoints conveys little information on the validity of the surrogate. In addition, in the normal setting, the magnitude of the ICA does not depend on the mean of the outcomes. The latter implies that identifiable parameters such as mean responses and treatment effects provide no information on the validity of the surrogate. In the present work it is shown that this is fundamentally different in the binary case. We demonstrate that the observed association between the outcomes as well as the success rates in both treatment groups are quite predictive for the ICA. It is shown that finding a good surrogate will be more likely when the association between the endpoints is large, there are sizeable treatment effects and the success rates for both endpoints are similar in both treatment groups. These results are demonstrated using extensive simulations and illustrated on a case study in multi-drug resistant tuberculosis.     

Assessing a surrogate predictive value: a causal inference approach

Several methods have been developed for the evaluation of surrogate endpoints within the causal‐inference and meta‐analytic paradigms. In both paradigms, much effort has been made to assess the capacity of the surrogate to predict the causal treatment effect on the true endpoint. In the present work, the so‐called surrogate predictive function (SPF) is introduced for that purpose, using potential outcomes. The relationship between the SPF and the individual causal association, a new metric of surrogacy recently proposed in the literature, is studied in detail. It is shown that the SPF, in conjunction with the individual causal association, can offer an appealing quantification of the surrogate predictive value. However, neither the distribution of the potential outcomes nor the SPF are identifiable from the data. These identifiability issues are tackled using a two‐step procedure. In the first step, the region of the parametric space of the distribution of the potential outcomes, compatible with the data at hand, is geometrically characterized. Further, in a second step, a Monte Carlo approach is used to study the behavior of the SPF on the previous region. The method is illustrated using data from a clinical trial involving schizophrenic patients and a newly developed and user friendly R package Surrogate is provided to carry out the validation exercise. Copyright © 2016 John Wiley & Sons, Ltd.     

A unifying approach for surrogate marker validation based on Prentice's criteria

Part of the recent literature on the evaluation of surrogate endpoints starts from a multi-trial approach which leads to a definition of validity in terms of the quality of both trial-level and individual-level association between a potential surrogate and a true endpoint, Buyse et al. These authors proposed their methodology based on the simplest cross-sectional case in which both the surrogate and the true endpoint are continuous and normally distributed. Different variations to this theme have been implemented for binary responses, times to event, combinations of binary and continuous endpoints, etc. However, a drawback of this methodology is that different settings have led to different definitions to quantify the association at the individual-level. In the longitudinal setting; Alonso et al. defined a class of canonical correlation functions that can be used to study surrogacy at the trial and individual-level. In the present work, we propose a new approach to evaluate surrogacy in the repeated measurements framework, we also show the connection between this proposal and the previous ones reported in the literature. Finally, we extend this concept to the non-normal case using the so-called 'likelihood reduction factor' (LRF) a new validation measure based on some of the Prentice's criteria. We apply the previous methodology using data from two clinical studies in psychiatry and ophthalmology.     

Bayesian estimation of the proportion of treatment effect captured by a surrogate marker

Surrogate endpoints in clinical trials are biological markers or events observable earlier than the clinical endpoints (such as death) that are actually of primary interest. The "proportion of treatment effect captured" by a surrogate endpoint (PTE) is a frequentist measure intended to address the question of whether trials based on a surrogate endpoint reach the same conclusions as would have been reached using the true endpoint. The question of inferential interest is whether PTE for a given marker exceeds some threshold value, say 0.5. Calculating PTE requires fitting two different models to the same data. We develop a Markov chain Monte Carlo based method for estimating the Bayesian posterior distribution of PTE. The new method conditions on the truth of a single model. Obtaining the full posterior distribution enables direct statements such as "the posterior probability that PTE >0.5 is 0.085". Furthermore, credible sets do not depend on asymptotic approximations and can be computed using data sets for which the frequentist methods may be inaccurate or even impossible to apply. We illustrate with Bayesian proportional hazards models for clinical trial data. As a by-product of developing the Bayesian method, we show that the frequentist estimate of PTE also may be computed from quantities in a single model and calculate frequentist confidence intervals for PTE that tend to be narrower than those produced by standard methods but that provide equally good coverage.     

Hard,harder, hardest: principal stratification,statistical identifiability,and the inherent difficulty of finding surrogate endpoints

In many areas of clinical investigation there is great interest in identifying and validating surrogate endpoints, biomarkers that can be measured a relatively short time after a treatment has been administered and that can reliably predict the effect of treatment on the clinical outcome of interest. However, despite dramatic advances in the ability to measure biomarkers, the recent history of clinical research is littered with failed surrogates. In this paper, we present a statistical perspective on why identifying surrogate endpoints is so difficult. We view the problem from the framework of causal inference, with a particular focus on the technique of principal stratification (PS), an approach which is appealing because the resulting estimands are not biased by unmeasured confounding. In many settings, PS estimands are not statistically identifiable and their degree of non-identifiability can be thought of as representing the statistical difficulty of assessing the surrogate value of a biomarker. In this work, we examine the identifiability issue and present key simplifying assumptions and enhanced study designs that enable the partial or full identification of PS estimands. We also present example situations where these assumptions and designs may or may not be feasible, providing insight into the problem characteristics which make the statistical evaluation of surrogate endpoints so challenging.     

Surrogate endpoints in clinical trials: cardiovascular diseases

A surrogate endpoint in a cardiovascular clinical trial is defined as endpoint measured in lieu of some other so-called 'true' endpoint. A surrogate is especially useful if it is easily measured and highly correlated with the true endpoint. Often the 'true' endpoint is one with clinical importance to the patient, for example, mortality or a major clinical outcome, while a surrogate is one biologically closer to the process of disease, for example, ejection fraction. Use of the surrogate can often lead to dramatic reductions in sample size and much shorter studies than use of the true endpoint. We discuss several problems common in trials with surrogate endpoints. Most important is the effect of missing data, especially in the face of informative censoring. Possible solutions are the assignment of scores or formal penalties to missing data.     

Surrogate endpoints in clinical trials: definition and operational criteria

I discuss the idea of using surrogate endpoints in the context of clinical trials to compare two or more treatments or interventions in respect to some 'true' endpoint, typically a disease occurrence. In order that treatment comparison based on a surrogate response variable have a meaningful implication for the corresponding true endpoint treatment comparison, a rather restrictive criterion is proposed for use of the adjective 'surrogate'. Specifically, I propose that a surrogate for a true endpoint yield a valid test of the null hypothesis of no association between treatment and the true response. This criterion essentially requires the surrogate variable to 'capture' any relationship between the treatment and the true endpoint, a notion that can be operationalized by requiring the true endpoint rate at any follow-up time to be independent of treatment, given the preceding history of the surrogate variable. I then discuss this operational criterion in the examples of the accompanying papers and in the setting of trials aimed at the primary and secondary prevention of cancer.