Adequate sample size for developing prediction models is not simply related to events per variable

ObjectivesThe choice of an adequate sample size for a Cox regression analysis is generally based on the rule of thumb derived from simulation studies of a minimum of 10 events per variable (EPV). One simulation study suggested scenarios in which the 10 EPV rule can be relaxed. The effect of a range of binary predictors with varying prevalence, reflecting clinical practice, has not yet been fully investigated.Study Design and SettingWe conducted an extended resampling study using a large general-practice data set, comprising over 2 million anonymized patient records, to examine the EPV requirements for prediction models with low-prevalence binary predictors developed using Cox regression. The performance of the models was then evaluated using an independent external validation data set. We investigated both fully specified models and models derived using variable selection.ResultsOur results indicated that an EPV rule of thumb should be data driven and that EPV ≥ 20 ​ generally eliminates bias in regression coefficients when many low-prevalence predictors are included in a Cox model.ConclusionHigher EPV is needed when low-prevalence predictors are present in a model to eliminate bias in regression coefficients and improve predictive accuracy.     

A pharmacodynamic analysis method to determine the relative importance of drug concentration and treatment time on effect

Purpose: The pharmacodynamics of most drugs follow the empirical relationship, Cⁿ × T=h, where C is drug concentration, T is exposure time and h is drug exposure constant. The value of n indicates the relative importance of C and T in determining the effect. An n value greater than 1.0 indicates that for two infusions that produce the same C × T, a short infusion that delivers high concentrations over a short duration will produce a greater Cⁿ × T and therefore a greater effect, compared to a long infusion that delivers lower concentrations. The reverse is true for an n value less than 1.0 and would support the use of a slow infusion. Hence, it is important to determine the n values and whether the n value significantly differs from 1.0. This report describes a three-step method for this purpose. Methods: First, we obtained experimental data on the relationship between drug concentration, treatment time and effect, and analyzed the data with a three-dimensional surface response method to obtain the pharmacodynamic model parameters and the magnitude of data variability. The experiments used mitomycin C and two human cancer cell lines, i.e. bladder RT4 and pharynx FaDu cells. The n values obtained from four experiments ranged from 1.04 to 1.16 for FaDu cells and from 1.14 to 1.46 for RT4 cells. The variability in the effect data decreased from 11.9% at 0% effect to 6.14% at 100% effect. Second, these results were used with Monte Carlo simulations to generate 100 concentration-time-effect data sets, which contained randomly and normally distributed data variability comparable to the experimentally observed variability, for each experimentally determined n value. This is analogous to performing 100 experiments under the same experimental conditions. Third, we analyzed the simulated data sets to obtain 100 estimated n values. The frequency with which these estimated n values fell above or below 1.0 indicated the probability that the experimentally determined n value used in the Monte Carlo simulations was truly different from 1.0. We defined this frequency for individual experiments as F_one, and calculated the overall probability for multiple experiments (F_multiple). A probability of greater than 97.5% (i.e. P < 0.05 for a two-tailed test) was considered statistically significant. Results: Analysis of the mitomycin C pharmacodynamic data yielded F_one and F_multiple of 99% to 100% for FaDu and RT4 cells, indicating that the n values for these cells were significantly higher than 1.0. A comparison of the statistical significance of the n value analyzed by the three-step pharmacodynamic analysis method, a conventional statistical method such as the Student's t-test and nonlinear regression analysis, indicated two advantages for the pharmacodynamic method: fewer experiments were required (theoretically only one experiment with three replicates would be sufficient) and a higher statistical significance of the n value was obtained. Conclusions: In summary, the three-step pharmacodynamic study design and analysis method can be used to define the relative importance of drug concentration and treatment time on drug effect. Received: 19 May 1999 / Accepted: 15 September 1999     

Classifier performance estimation under the constraint of a finite sample size: Resampling schemes applied to neural network classifiers

In a practical classifier design problem the sample size is limited, and the available finite sample needs to be used both to design a classifier and to predict the classifier’s performance for the true population. Since a larger sample is more representative of the population, it is advantageous to design the classifier with all the available cases, and to use a resampling technique for performance prediction. We conducted a Monte Carlo simulation study to compare the ability of different resampling techniques in predicting the performance of a neural network (NN) classifier designed with the available sample. We used the area under the receiver operating characteristic curve as the performance index for the NN classifier. We investigated resampling techniques based on the cross-validation, the leave-one-out method, and three different types of bootstrapping, namely, the ordinary, .632, and .632+ bootstrap. Our results indicated that, under the study conditions, there can be a large difference in the accuracy of the prediction obtained from different resampling methods, especially when the feature space dimensionality is relatively large and the sample size is small. Although this investigation is performed under some specific conditions, it reveals important trends for the problem of classifier performance prediction under the constraint of a limited data set.     

More on the means comparison with unequal variances problem

Scheffe gave an exact solution to the problem of comparing two means from normal populations with unequal variances that is useful for general analysis of variance problems. The behavior of the usual t-statistic that assumes equal variances is contrasted to Satterthwaite's approximate t- statistic and Scheffe's method. An interesting relationship is uncovered between Scheffe's and Satterthwaite's solutions.     

Subgroup Identification in Clinical Trials by Stochastic SIDEScreen Methods

ABSTRACT

Subgroup identification for personalized medicine has become very popular in the last decade. Efficient recursive partitioning procedures adapted from machine learning are natural approaches for performing subgroup identification based on pre-defined biomarkers since they provide subgroups as terminal nodes in the decision tree. However, recursive partitioning is also known as a potentially unstable procedure with results being quite sensitive to normal sampling variability in the data. One common approach, borrowed from ensemble learning, to overcome such instability is application of recursive partitioning to multiple data sets sampled from the observed data followed by averaging the results over the collection of subgroups.

This article proposes an alternative approach to subgroup identification in clinical trials that first evaluates the predictive strength of biomarkers based on variable importance and then applies recursive partitioning to the biomarkers with the highest variable importance scores. A deterministic version of this idea was implemented in the Adaptive SIDEScreen method that generates a collection of patient subgroups by retaining multiple candidate splits of each parent group by different biomarkers (Lipkovich and Dmitrienko 2014a Lipkovich, I., and Dmitrienko, A. (2014a), “Strategies for Identifying Predictive Biomarkers and Subgroups With Enhanced Treatment Effect in Clinical Trials Using SIDES,” Journal of Biopharmaceutical Statistics, 24, 130–153.[Taylor & Francis Online], [Web of Science ®] , [Google Scholar], 2014b ——— (2014b), “Biomarker Identification in Clinical Trials,” in Clinical and Statistical Considerations in Personalized Medicine, eds. C. Carini, S. Menon, and M. Chang, New York: Chapman and Hall. [Google Scholar]). Now, we extend the Adaptive SIDEScreen and introduce the Stochastic SIDEScreen method. The key idea is to introduce randomness in the subgroup generation process, borrowing from bagging methods, to produce a broader collection of subgroups. Specifically, the SIDES method, where the most promising biomarkers are selected for each parent group from a set of candidate biomarkers, is applied to multiple bootstrap samples of the data. This new approach leads to a more reliable biomarker selection process, which is especially important for smaller, early phase studies when biomarker selection is typically carried out. The method is illustrated using clinical trial examples.     

重采样技术在中老年居民糖尿病不平衡数据分类中的应用

目的 利用重采样技术提高我国中老年居民糖尿病不平衡数据的分类预测效果。方法 采用随机欠采样、随机过采样、合成少数类过采样(synthetic minority oversampling technique, SMOTE)以及自适应合成抽样(adaptive synthetic sampling, ADASYN)四种重采样技术处理CHARLS数据库中糖尿病不平衡数据,比较重采样前后logistic回归、支持向量机、随机森林的分类性能,采用G-means和AUC评价模型的预测效果。结果 对CHARLS糖尿病不平衡数据集,logistic回归、支持向量机、随机森林模型的G-means分别为0.222 7、0、0,AUC分别为0.761 2、0.736 3、0.742 9,logistic回归模型显著优于支持向量机,模型准确率(χ²=1 231.501,P<0.001)及AUC值(Z=2.634, P=0.028)的差异均具有统计学意义。四种重采样技术处理后模型的G-means均提高,特别是SMOTE和ADASYN技术;此外,随机欠采样不能显著提高logistic回...     

Personalized Medicine: Four Perspectives of Tailored Medicine

With the rapid advancement of scientific understanding in the medical field, everyday use of personalized medicine appears within our grasp. Unfortunately, there are still challenges to overcome. In many therapeutic areas, there remains a lack of deep understanding of disease processes and treatments. Additionally, drug development proceeds over a 5–10 year timeframe, during which time knowledge of treatments and potential predictive biomarkers continues to evolve. For successful development of a drug that is tailored to a biomarker-defined patient population and approved by regulators for such use, employment of appropriate statistical design and analysis methods is paramount. Consequently, statisticians can play a leading role in transforming the practice of medicine to a more personalized approach. We describe four perspectives in clinical development, together with examples of each, and discuss how to approach the problem of demonstrating that a treatment works better in a biomarker-defined subgroup of patients than in its complementary subgroup. The four perspectives provide a framework for design of clinical trials and subsequent analyses as they relate to clinical development. Subgroup identification is described as a controlled, disciplined search for finding the right patient for treatment and is distinguished from traditional, exploratory subgroup analysis.     

Cross-validation of matching correlation analysis by resampling matching weights

The strength of association between a pair of data vectors is represented by a nonnegative real number, called matching weight. For dimensionality reduction, we consider a linear transformation of data vectors, and define a matching error as the weighted sum of squared distances between transformed vectors with respect to the matching weights. Given data vectors and matching weights, the optimal linear transformation minimizing the matching error is solved by the spectral graph embedding of Yan et al. (2007). This method is a generalization of the canonical correlation analysis, and will be called as matching correlation analysis (MCA). In this paper, we consider a novel sampling scheme where the observed matching weights are randomly sampled from underlying true matching weights with small probability, whereas the data vectors are treated as constants. We then investigate a cross-validation by resampling the matching weights. Our asymptotic theory shows that the cross-validation, if rescaled properly, computes an unbiased estimate of the matching error with respect to the true matching weights. Existing ideas of cross-validation for resampling data vectors, instead of resampling matching weights, are not applicable here. MCA can be used for data vectors from multiple domains with different dimensions via an embarrassingly simple idea of coding the data vectors. This method will be called as cross-domain matching correlation analysis (CDMCA), and an interesting connection to the classical associative memory model of neural networks is also discussed.     

基于多媒体技术的快速并行体绘制

体绘制的成像速度一直是影响其发展的关键因素。我们提出了一种基于多媒体单指令多数据(single instruction multiple data,SIMD)技术的加速算法。在保证成像质量的前提下,通过采用MMX、SSE、SSE2指令,分别加速光线扫描、采样和坐标变换等部分,使得光线跟踪算法的成像速度提高了3～6倍。实验表明,该算法具有成像速度快、成像稳定等显著优势,具有很高的应用价值。     

The tapered block bootstrap for general statistics from stationary sequences

In this paper, we define and study a new block bootstrap variation, the tapered block bootstrap, that is applicable in the general case of approximately linear statistics, and constitutes an improvement over the original block bootstrap of 15 . The asymptotic validity, and the favorable bias properties of the tapered block bootstrap are shown in two important cases: smooth functions of means, and M ‐estimators. The important practical issues of optimally choosing the window shape and the block size are addressed in detail, while some finite‐sample simulations are also presented.