Effective sample sizes for confidence intervals for survival probabilities

We examine various methods to estimate the effective sample size for construction of confidence intervals for survival probabilities. We compare the effective sample sizes of Cutler and Ederer and Peto et al., as well as a modified Cutler-Ederer effective sample size. We investigate the use of these effective sample sizes in the common situation of many censored observations that intervene between the time point of interest and the last death before this time. We note that there is no a priori reason to treat upper and lower confidence intervals in a symmetric fashion since censored survival data are by nature asymmetric. We recommend the use of the Cutler-Ederer effective sample size in construction of upper confidence intervals and the Peto effective sample size in construction of lower confidence intervals. Two examples with real data demonstrate the differences between confidence intervals formed with different effective sample sizes. This study also illustrates the need for caution in the application of simulation studies to real problems.     

Finite sample pointwise confidence intervals for a survival distribution with right‐censored data

We review and develop pointwise confidence intervals for a survival distribution with right‐censored data for small samples, assuming only independence of censoring and survival. When there is no censoring, at each fixed time point, the problem reduces to making inferences about a binomial parameter. In this case, the recently developed beta product confidence procedure (BPCP) gives the standard exact central binomial confidence intervals of Clopper and Pearson. Additionally, the BPCP has been shown to be exact (gives guaranteed coverage at the nominal level) for progressive type II censoring and has been shown by simulation to be exact for general independent right censoring. In this paper, we modify the BPCP to create a ‘mid‐p’ version, which reduces to the mid‐p confidence interval for a binomial parameter when there is no censoring. We perform extensive simulations on both the standard and mid‐p BPCP using a method of moments implementation that enforces monotonicity over time. All simulated scenarios suggest that the standard BPCP is exact. The mid‐p BPCP, like other mid‐p confidence intervals, has simulated coverage closer to the nominal level but may not be exact for all survival times, especially in very low censoring scenarios. In contrast, the two asymptotically‐based approximations have lower than nominal coverage in many scenarios. This poor coverage is due to the extreme inflation of the lower error rates, although the upper limits are very conservative. Both the standard and the mid‐p BPCP methods are available in our bpcp R package. Published 2016. This article is US Government work and is in the public domain in the USA.     

Simultaneous non-parametric confidence intervals for survival probabilities from censored data

In medical studies with censored data Kaplan and Meier's product limit estimator has frequent use as the estimate of the survival function. Simultaneous confidence intervals for the survival function at various time points constitute a useful addition to the analysis. This study compares several such methods. We consider in a simulation investigation two whole curve confidence bands and four methods based on the Bonferroni inequality. The results show that three Bonferroni-type methods are essentially equivalent, all being better than the other methods when the number of time points is small (3 or 5).     

Interval-censored survival time data: confidence intervals for the non-parametric survivor function

Survival data are described as interval censored when the failure time is not measured exactly but is known only to have occurred within a defined interval. In this paper, we describe and assess three methods for calculating pointwise confidence intervals for the non-parametric survivor function estimated from interval-censored data: the first based on the full information matrix, the second a modification of this approach involving deletion of rows and columns of the information matrix corresponding to zero estimates prior to inversion and the third based on likelihood ratio inference. In a simulation study the likelihood ratio method gave the most accurate confidence intervals with coverage consistently close to the nominal level of 95 per cent.     

Covariates-dependent confidence intervals for the difference or ratio of two median survival times

In this paper, we are concerned with the estimation of the discrepancy between two treatments when right-censored survival data are accompanied with covariates. Conditional confidence intervals given the available covariates are constructed for the difference between or ratio of two median survival times under the unstratified and stratified Cox proportional hazards models, respectively. The proposed confidence intervals provide the information about the difference in survivorship for patients with common covariates but in different treatments. The results of a simulation study investigation of the coverage probability and expected length of the confidence intervals suggest the one designed for the stratified Cox model when data fit reasonably with the model. When the stratified Cox model is not feasible, however, the one designed for the unstratified Cox model is recommended. The use of the confidence intervals is finally illustrated with a HIV+ data set.     

Regaining confidence in confidence intervals for the mean treatment effect

In many experiments, it is necessary to evaluate the effectiveness of a treatment by comparing the responses of two groups of subjects. This evaluation is often performed by using a confidence interval for the difference between the population means. To compute the limits of this confidence interval, researchers usually use the pooled t formulas, which are derived by assuming normally distributed errors. When the normality assumption does not seem reasonable, the researcher may have little confidence in the confidence interval because the actual one‐sided coverage probability may not be close to the nominal coverage probability. This problem can be avoided by using the Robbins–Monro iterative search method to calculate the limits. One problem with this iterative procedure is that it is not clear when the procedure produces a sufficiently accurate estimate of a limit. In this paper, we describe a multiple search method that allows the user to specify the accuracy of the limits. We also give guidance concerning the number of iterations that would typically be needed to achieve a specified accuracy. This multiple iterative search method will produce limits for one‐sided and two‐sided confidence intervals that maintain their coverage probabilities with non‐normal distributions. Copyright © 2014 John Wiley & Sons, Ltd.     

Mid-p confidence intervals for the poisson expectation

Conventionally a confidence interval (CI) for the standardized mortality ratio is set using the conservative CI for a Poisson expectation, μ. Employing the mid-P argument we present alternative CIs that are shorter than the conventional ones. The mid-P intervals do not guarantee the nominal confidence level, but the true coverage probability is only lower than the nominal level for a few short ranges of μ. The implications for mid-P confidence intervals of various proposed definitions of two-sided tests for discrete data are discussed.     

Ensemble confidence intervals for binomial proportions

We propose two measures of performance for a confidence interval for a binomial proportion p: the root mean squared error and the mean absolute deviation. We also devise a confidence interval for p based on the actual coverage function that combines several existing approximate confidence intervals. This “Ensemble” confidence interval has improved statistical properties over the constituent confidence intervals. Software in an R package, which can be used in devising and assessing these confidence intervals, is available on CRAN.     

Likelihood-based confidence intervals for a log-normal mean

To construct a confidence interval for the mean of a log-normal distribution in small samples, we propose likelihood-based approaches - the signed log-likelihood ratio and modified signed log-likelihood ratio methods. Extensive Monte Carlo simulation results show the advantages of the modified signed log-likelihood ratio method over the signed log-likelihood ratio method and other methods. In particular, the modified signed log-likelihood ratio method produces a confidence interval with a nearly exact coverage probability and highly accurate and symmetric error probabilities even for extremely small sample sizes. We then apply the methods to two sets of real-life data.     

Practicable confidence intervals for current status data

Although confidence intervals (CIs) for binary isotonic regression and current status survival data have been well studied theoretically, their practical application has been limited, in part because of poor performance in small samples and in part because of computational difficulties. Ghosh, Banerjee, and Biswas (2008, Biometrics 64 , 1009‐1017) described three approaches to constructing CIs: (i) the Wald‐based method; (ii) the subsampling‐based method; and (iii) the likelihood‐ratio test (LRT)‐based method. In simulation studies, they found that the subsampling‐based method and LRT‐based method tend to have better coverage probabilities than a simple Wald‐based method that may perform poorly in realistic sample sizes. However, software implementing these approaches is currently unavailable. In this article, we show that by using transformations, simple Wald‐based CIs can be improved with small and moderate sample sizes to have competitive performance with LRT‐based method. Our simulations further show that a simple nonparametric bootstrap gives approximately correct CIs for the data generating mechanisms that we consider. We provide an R package that can be used to compute the Wald‐type and the bootstrap CIs and demonstrate its practical utility with two real data analyses. Copyright © 2012 John Wiley & Sons, Ltd.     

Non-parametric bootstrap confidence intervals for the intraclass correlation coefficient

The intraclass correlation coefficient rho plays a key role in the design of cluster randomized trials. Estimates of rho obtained from previous cluster trials and used to inform sample size calculation in planned trials may be imprecise due to the typically small numbers of clusters in such studies. It may be useful to quantify this imprecision. This study used simulation to compare different methods for assigning bootstrap confidence intervals to rho for continuous outcomes from a balanced design. Data were simulated for combinations of numbers of clusters (10, 30, 50), intraclass correlation coefficients (0.001, 0.01, 0.05, 0.3) and outcome distributions (normal, non-normal continuous). The basic, bootstrap-t, percentile, bias corrected and bias corrected accelerated bootstrap intervals were compared with new methods using the basic and bootstrap-t intervals applied to a variance stabilizing transformation of rho. The standard bootstrap methods provided coverage levels for 95 per cent intervals that were markedly lower than the nominal level for data sets with only 10 clusters, and only provided close to 95 per cent coverage when there were 50 clusters. Application of the bootstrap-t method to the variance stabilizing transformation of rho improved upon the performance of the standard bootstrap methods, providing close to nominal coverage.     

Simple improved confidence intervals for comparing matched proportions

For binary matched-pairs data, this article discusses interval estimation of the difference of probabilities and an odds ratio for comparing 'success' probabilities. We present simple improvements of the commonly used Wald confidence intervals for these parameters. The improvement of the interval for the difference of probabilities is to add two observations to each sample before applying it. The improvement for estimating an odds ratio transforms a confidence interval for a single proportion.     

Repeated confidence intervals for adaptive group sequential trials

This paper proposes a method for computing conservative confidence intervals for a group sequential test in which an adaptive design change is made one or more times over the course of the trial. The key idea, due to Müller and Sch?fer (Biometrics 2001; 57:886-891), is that by preserving the null conditional rejection probability of the remainder of the trial at the time of each adaptive change, the overall type I error rate, taken unconditionally over all possible design modifications, is also preserved. We show how this principle may be extended to construct one-sided confidence intervals by applying the idea to a sequence of dual tests derived from the repeated confidence intervals (RCIs) proposed by Jennison and Turnbull (J. Roy. Statist. Soc. B 1989; 51:301-361). These adaptive RCIs, such as their classical counterparts, have the advantage that they preserve the desired coverage probability even if the pre-specified stopping rule is over-ruled. The statistical methodology is explored by simulations and is illustrated by an application to a clinical trial of deep brain stimulation for Parkinson's disease.     

Standard errors and confidence intervals for variable importance in random forest regression,classification, and survival

Random forests are a popular nonparametric tree ensemble procedure with broad applications to data analysis. While its widespread popularity stems from its prediction performance, an equally important feature is that it provides a fully nonparametric measure of variable importance (VIMP). A current limitation of VIMP, however, is that no systematic method exists for estimating its variance. As a solution, we propose a subsampling approach that can be used to estimate the variance of VIMP and for constructing confidence intervals. The method is general enough that it can be applied to many useful settings, including regression, classification, and survival problems. Using extensive simulations, we demonstrate the effectiveness of the subsampling estimator and in particular find that the delete-d jackknife variance estimator, a close cousin, is especially effective under low subsampling rates due to its bias correction properties. These 2 estimators are highly competitive when compared with the .164 bootstrap estimator, a modified bootstrap procedure designed to deal with ties in out-of-sample data. Most importantly, subsampling is computationally fast, thus making it especially attractive for big data settings.     

Prediction intervals for survival data

This paper concerns large sample prediction intervals for the survival times of a future sample based on an initial sample of censored survival data. Simple procedures are developed for obtaining non-parametric and exponential prediction intervals for the future sample quantiles; the non-parametric interval results from inversion of an appropriate test statistic. A simulation study performed under various conditions evaluates the accuracy of the proposed intervals. An adjuvant chemotherapy study of breast cancer patients illustrates the methodology.     

Bootstrap confidence intervals for the sensitivity of a quantitative diagnostic test

We examine bootstrap approaches to the analysis of the sensitivity of quantitative diagnostic test data. Methods exist for inference concerning the sensitivity of one or more tests for fixed levels of specificity, taking into account the variability in the sensitivity due to variability in the test values for normal subjects. However, parametric methods do not adequately account for error, particularly when the data are non-normally distributed, and non-parametric methods have low power. We implement bootstrap methods for confidence limits for the sensitivity of a test for a fixed specificity and demonstrate that under certain circumstances the bootstrap method gives more accurate confidence intervals than do other methods, while it performs at least as well as other methods in many standard situations.     

Sample sizes for constructing confidence intervals and testing hypotheses

Although estimation and confidence intervals have become popular alternatives to hypothesis testing and p-values, statisticians usually determine sample sizes for randomized clinical trials by controlling the power of a statistical test at an appropriate alternative, even those statisticians who recommend the use of confidence intervals for inference. There is merit in achieving consistency in the techniques for data analysis and sample size determination. To that end, this paper compares sample size determination with use of the length of the confidence interval with that obtained by control of power.