Classifier performance estimation under the constraint of a finite sample size: Resampling schemes applied to neural network classifiers

In a practical classifier design problem the sample size is limited, and the available finite sample needs to be used both to design a classifier and to predict the classifier’s performance for the true population. Since a larger sample is more representative of the population, it is advantageous to design the classifier with all the available cases, and to use a resampling technique for performance prediction. We conducted a Monte Carlo simulation study to compare the ability of different resampling techniques in predicting the performance of a neural network (NN) classifier designed with the available sample. We used the area under the receiver operating characteristic curve as the performance index for the NN classifier. We investigated resampling techniques based on the cross-validation, the leave-one-out method, and three different types of bootstrapping, namely, the ordinary, .632, and .632+ bootstrap. Our results indicated that, under the study conditions, there can be a large difference in the accuracy of the prediction obtained from different resampling methods, especially when the feature space dimensionality is relatively large and the sample size is small. Although this investigation is performed under some specific conditions, it reveals important trends for the problem of classifier performance prediction under the constraint of a limited data set.     

Expression of the vascular permeability factor/vascular endothelial growth factor gene in central nervous system neoplasms.

Expression of the vascular permeability factor/vascular endothelial growth factor (VEGPF) gene was investigated in human central nervous system (CNS) neoplasms and normal brain. Adsorption of capillary permeability activity from human glioblastoma multiforme (GBM) cell conditioned medium and GBM cyst fluids by anti-VEGPF antibodies demonstrated that VEGPF is secreted by GBM cells and is present in sufficient quantities in vivo to induce vascular permeability. Cloning and sequencing of polymerase chain reaction-amplified GBM and normal brain cDNA demonstrated three forms of the VEGPF coding region (567, 495, and 363 nucleotides), corresponding to mature polypeptides of 189, 165, and 121 amino acids, respectively. VEGPF mRNA levels in CNS tumors vs. normal brain were investigated by the RNase protection assay. Significant elevation of VEGPF gene expression was observed in 81% (22/27) of the highly vascular and edema-associated CNS neoplasms (6/8 GBM, 8/8 capillary hemangioblastomas, 6/7 meningiomas, and 2/4 cerebral metastases). In contrast, only 13% (2/15) of those CNS tumors that are not commonly associated with significant neovascularity or cerebral edema (2/10 pituitary adenomas and 0/5 nonastrocytic gliomas) had significantly increased levels of VEGPF mRNA. The relative abundance of the forms of VEGPF mRNA was consistent in tumor and normal brain: VEGPF495 > VEGPF363 > VEGPF567. In situ hybridization confirmed the presence of VEGPF mRNA in tumor cells and its increased abundance in capillary hemangioblastomas. Our results suggest a significant role for VEGPF in the development of CNS tumor neovascularity and peritumoral edema.     

Targeting PSMA with a Cu-64 Labeled Phosphoramidate Inhibitor for PET/CT Imaging of Variant PSMA-Expressing Xenografts in Mouse Models of Prostate Cancer

Purpose

Prostate-specific membrane antigen (PSMA) is highly up-regulated in prostate tumor cells, providing an ideal target for imaging applications of prostate cancer. CTT-1297 (IC₅₀?=?27 nM) is an irreversible phosphoramidate inhibitor of PSMA that has been conjugated to the CB-TE1K1P chelator for incorporation of Cu-64. The resulting positron emission tomography (PET) agent, [⁶⁴Cu]ABN-1, was evaluated for selective uptake both in vitro and in vivo in PSMA-positive cells of varying expression levels. The focus of this study was to assess the ability of [⁶⁴Cu]ABN-1 to detect and distinguish varying levels of PSMA in a panel of prostate tumor-bearing mouse models.

Procedures

CTT-1297 was conjugated to the CB-TE1K1P chelator using click chemistry and radiolabeled with Cu-64. Internalization and binding affinity of [⁶⁴Cu]ABN-1 was evaluated in the following cell lines having varying levels of PSMA expression: LNCaP late-passage?>?LNCaP early passage?≈?C4-2B?>?CWR22rv1 and PSMA-negative PC-3 cells. PET/X-ray computed tomography imaging was performed in NCr nude mice with subcutaneous tumors of the variant PSMA-expressing cell lines.

Results

[⁶⁴Cu]ABN-1 demonstrated excellent uptake in PSMA-positive cells in vitro, with ～80 % internalization at 4 h for each PSMA-positive cell line with uptake (fmol/mg) correlating to PSMA expression levels. The imaging data indicated significant tumor uptake in all models. The biodistribution for late-passage LNCaP (highest PSMA expression) demonstrated the highest specific uptake of [⁶⁴Cu]ABN-1 with tumor-to-muscle and tumor-to-blood ratios of 30?±?11 and 21?±?7, respectively, at 24 h post-injection. [⁶⁴Cu]ABN-1 cleared through all tissues except for PSMA-positive kidneys.

Conclusion

[⁶⁴Cu]ABN-1 demonstrated selective uptake in PSMA-positive cells and tumors, which correlated to the level of PSMA expression. The data reported herein suggest that [⁶⁴Cu]ABN-1 will selectively target and image variant PSMA expression and in the future will serve as a non-invasive method to follow the progression of prostate cancer in men.

     

Specific Agglutinability of Erythrocytes from Whole Blood Stored at 4 C

When whole blood is supplemented with adenine, the erythrocytes are suitable for transfusion after at least 35 days of storage at 4C, but information is not available as to whether such cells remain satisfactory for blood typing and pre transfusion tests for evidence of incompatibility. Accordingly, we designed experiments to evaluate by AutoAnalyzer the specific agglutinability of erythrocytes obtained from whole blood stored at 4C. Seven normal volunteer donors of both sexes, aged between 22 and 31 years, were chosen and bled periodically so that, over a three-day testing period, all blood specimens from each donor could be evaluated. Each donor provided at each bleeding sufficient blood to permit aliquots of 20 ml to be stored as clotted blood and 8 ml to be stored as citrated whole blood. Four kinds of citrate solution were used: ACD Formula A, CPD, ACD supplemented with adenine, and CPD supplemented with adenine. All clotted specimens were tested after 0, 1, 2, 3, 4, and 5 weeks of storage, while all citrated specimens were tested after 0, 1, 2, 3, 4, 5, 6, 8, and 10 weeks of storage. Six blood group systems were used for evaluation with the following reagents: anti-A or anti-B, anti-Rhl (Rh_o or D), anti-K2 (k or Cellano), anti-Jk^a, anti-Fy^a or anti-Fy^b, and anti-M. Each reagent was used at a dilution to support from 20 to 80 per cent agglutination with freshly drawn blood, and each reagent was tested under two conditions: low ionic and normal ionic. The results disclosed loss of specific agglutinability in association with the time of storage of whole blood at 4 C. The onset and degree of loss depended both on the kind of test used (normal ionic tests showed loss 2.8 weeks earlier than did low ionic tests) and the condition in which the blood was stored (loss occurred sooner and then progressed more with clotted blood than with citrated blood). Average losses of 25, 50, and 75 per cent were observed at 1.1, 2.1, and 4.0 weeks of storage for normal ionic tests of clotted blood, and at 4.1, 6.0, and 7.6 weeks for similar tests of citrated blood. Loss of specific agglutinability during storage was significantly less for the red blood cells of some donors than for others, but systematic differences between specific blood-typing tests were not observed. ACD and CPD solutions behaved similarly, while adenine supplementation exerted a slight preservative effect after 6, 8, and 10 weeks of storage. Loss of specific hemagglutinability of stored cells was not due to loss of specific antigenic sites because stored cells were as effective as fresh cells for the absorption of anti-B, anti-Rhl, and anti-M. Loss might, however, be related to an increase in the accumulation of IgG, fibrinogen, and albumin at the surfaces of erythrocytes.     

Polarized secretion of transgene products from salivary glands in vivo

Previously (Kagami et al. Hum. Gene Ther. 1996;7:2177-2184) we have shown that salivary glands are able to secrete a transgene-encoded protein into serum as well as saliva. This result and other published data suggest that salivary glands may be a useful target site for vectors encoding therapeutic proteins for systemic delivery. The aim of the present study was to assess in vivo if transgene-encoded secretory proteins follow distinct, polarized sorting pathways as has been shown to occur "classically" in cell biological studies in vitro. Four first-generation, E1-, type 5 recombinant adenoviruses were used to deliver different transgenes to a rat submandibular cell line in vitro or to rat submandibular glands in vivo. Subsequently, the secretory distribution of the encoded proteins was determined. Luciferase, which has no signal peptide, served as a cell-associated, negative control and was used to correct for any nonspecific secretory protein release from cells. The three remaining transgene products tested, human tissue kallikrein (hK1), human growth hormone (hGH), and human alpha1-antitrypsin (halpha1AT), were predominantly secreted (>96%) in vitro. Most importantly, in vivo, after a parasympathomimetic secretory stimulus, both hK1 and hGH were secreted primarily in an exocrine manner into saliva. Conversely, halpha1AT was predominantly secreted into the bloodstream, i.e., in an endocrine manner. The aggregate results are consistent with the recognition of signals encoded within the transgenes that result in specific patterns of polarized protein secretion from rat submandibular gland cells in vivo.     

Malignant and benign breast masses on 3D US volumetric images: effect of computer-aided diagnosis on radiologist accuracy

PURPOSE: To retrospectively investigate the effect of using a custom-designed computer classifier on radiologists' sensitivity and specificity for discriminating malignant masses from benign masses on three-dimensional (3D) volumetric ultrasonographic (US) images, with histologic analysis serving as the reference standard. MATERIALS AND METHODS: Informed consent and institutional review board approval were obtained. Our data set contained 3D US volumetric images obtained in 101 women (average age, 51 years; age range, 25-86 years) with 101 biopsy-proved breast masses (45 benign, 56 malignant). A computer algorithm was designed to automatically delineate mass boundaries and extract features on the basis of segmented mass shapes and margins. A computer classifier was used to merge features into a malignancy score. Five experienced radiologists participated as readers. Each radiologist read cases first without computer-aided diagnosis (CAD) and immediately thereafter with CAD. Observers' malignancy rating data were analyzed with the receiver operating characteristic (ROC) curve. RESULTS: Without CAD, the five radiologists had an average area under the ROC curve (A(z)) of 0.83 (range, 0.81-0.87). With CAD, the average A(z) increased significantly (P = .006) to 0.90 (range, 0.86-0.93). When a 2% likelihood of malignancy was used as the threshold for biopsy recommendation, the average sensitivity of radiologists increased from 96% to 98% with CAD, while the average specificity for this data set decreased from 22% to 19%. If a biopsy recommendation threshold could be chosen such that sensitivity would be maintained at 96%, specificity would increase to 45% with CAD. CONCLUSION: Use of a computer algorithm may improve radiologists' accuracy in distinguishing malignant from benign breast masses on 3D US volumetric images.     

Relative deprivation in income and self-rated health in the United States

Absolute income is robustly associated with health status. Few studies have, however, examined if relative income is independently associated with health. We examined if, over and above the effects of absolute income, individual relative deprivation in income as well as position in the income hierarchy is associated with individual poor health in the U.S. Using three rounds of the Current Population Surveys (CPS), we analyzed the association between self-rated health (1 = fair/poor, 0 = otherwise) and the Yitzhaki index of relative deprivation in income and percentile position in the income hierarchy across 17 reference groups. Over and above the effects of absolute income, the odds ratio for reporting poor health among individuals in the highest quintile of relative deprivation compared to the lowest quintile ranged between 2.18 and 3.30, depending on the reference groups used. A 10 percentile increase in income position within reference groups was associated with an odds ratio of poor health of 0.89. Relative deprivation appeared to explain between 33 and 94% of the association between individual income and self-rated health. Relative deprivation in income is independently associated with poor health over and above the well established effects of absolute income on health. Relative deprivation may partly explain the association between income inequality and worse population health status.