On the use of the metric rm² as an effective tool for validation of QSAR models in computational drug design and predictive toxicology

Validation of quantitative structure-activity relationship (QSAR) models plays a key role for the selection of robust and predictive models that may be employed for further activity prediction of new molecules. Traditionally, QSAR models are validated based on classical metrics for internal (Q2) and external validation (R2 pred). Recently, it has been shown that for data sets with wide range of the response variable, these traditional metrics tend to achieve high values without truly reflecting absolute differences between the observed and predicted response values, as in both cases the reference for comparison of the predicted residuals is the deviations of the observed values from the training set mean. Roy et al. have recently developed a new parameter, modified r2 (rm2), which considers the actual difference between the observed and predicted response data without consideration of training set mean thereby serving as a more stringent measure for assessment of model predictivity compared to the traditional validation parameters (Q2 and R2 pred). The rm2 parameter has three different variants: (i) rm2 (LOO) for internal validation, (ii) rm2 (test) for external validation and (iii) rm2 (overall) for analyzing the overall performance of the developed model considering predictions for both internal and external validation sets. Thus, the rm2 metrics strictly judge the ability of a QSAR model to predict the activity/toxicity of untested molecules. The present review provides a survey of the development of different rm2 metrics followed by their applications in modeling studies for selection of the best QSAR models in different reports made by several workers.     

The physicochemical approach to drug design and discovery (QSAR)

The study of physicochemical parameters to correlate mathematically chemical structure with biological activity induced by sets of congeneric drugs is now generally referred to as QSAR (quantitative structure-activity relationships). The ways in which the QSAR paradigm are developing are becoming more varied and complex. Many kinds of parameters are under study in many different groups; various types of mathematical models have been proposed and are being evaluated. Drug researchers are turning more to enzyme modulation to control various biological processes. It is the study of enzyme-ligand reactions of enzymes whose x-ray crystallographic structure is known that affords us the means for developing a deeper understanding of QSAR and, at the same time, enhancing our ability to make drugs more selective for various forms of a given enzyme. The union of x-ray crystallography, moleculargraphics, and QSAR is one of the most exciting new areas of drug development. This report is an introduction to how QSAR is being used to gain insight into the interaction of drugs with macromolecules and macromolecular systems.     

Robust QSAR models using Bayesian regularized neural networks.

We describe the use of Bayesian regularized artificial neural networks (BRANNs) in the development of QSAR models. These networks have the potential to solve a number of problems which arise in QSAR modeling such as: choice of model; robustness of model; choice of validation set; size of validation effort; and optimization of network architecture. The application of the methods to QSAR of compounds active at the benzodiazepine and muscarinic receptors is illustrated.     

A comprehensive model for reproductive and developmental toxicity hazard identification: II. Construction of QSAR models to predict activities of untested chemicals

This report describes the construction, optimization and validation of a battery of quantitative structure-activity relationship (QSAR) models to predict reproductive and developmental (reprotox) hazards of untested chemicals. These models run with MC4PC software to predict seven general reprotox classes: male and female reproductive toxicity, fetal dysmorphogenesis, functional toxicity, mortality, growth, and newborn behavioral toxicity. The reprotox QSARs incorporate a weight of evidence paradigm using rats, mice, and rabbit reprotox study data and are designed to identify trans-species reprotoxicants. The majority of the reprotox QSARs exhibit good predictive performance properties: high specificity (>80%), low false positives (<20%), significant receiver operating characteristic (ROC) values (>2.00), and high coverage (>80%) in 10% leave-many-out validation experiments. The QSARs are based on 627-2023 chemicals and exhibited a wide applicability domain for FDA regulated organic chemicals for which they were designed. Experiments were also performed using the MC4PC multiple module prediction technology, and ROC statistics, and adjustments to the ratio of active to inactive (A/I ratio) chemicals in training data sets were made to optimize the predictive performance of QSAR models. Results revealed that an A/I ratio of approximately 40% was optimal for MC4PC. We discuss specific recommendations for the application of the reprotox QSAR battery.     

QSAR study of 5-arylidene-2,4-thiazolidinediones as aldose reductase inhibitors

The shortage of drugs currently available for the treatment of diabetic complications has aroused our interest for the search of new aldose reductase inhibitors (ARIs) endowed with more favorable biological properties. In response, quantitative structure–activity relationship (QSAR) study has been performed on a series of 5-arylidene-2,4-thiazolidinediones using the Fujita-Ban and the classical Hansch approach and molecular modeling studies employing AM1 calculations to gain structural insight into the binding mode of these molecules to the aldose reductase enzyme. The QSAR models were generated using 18 compounds. The predictive ability of the resulting QSAR models was evaluated employing the leave-one-out method of cross validation. Remarkably, the results obtained from the Fujita-Ban and Hansch approaches were in agreement with the molecular modeling studies. The QSAR analysis reported herein confirms that the presence of the carboxylic anionic head of the N-3 acetic chain is an important, albeit not essential, structural requisite to produce high levels of enzyme inhibition. Furthermore, the hydrophobic substitutions are conducive to this activity.     

Use of the index of ideality of correlation to improve predictive potential for biochemical endpoints

The CORAL software is a tool to build up quantitative structure-property/activity relationships (QSPRs/QSARs). The project of updated version of the CORAL software is discussed in terms of practical applications for building up various models. The updating is the possibility to improve the predictive potential of models using the so-called Index of Ideality of Correlation (IIC) as a criterion of the predictive potential for QSPR/QSAR models. Efficacy of the IIC is examined with three examples of building up QSARs: (i) models for anticancer activity; (ii) models for mutagenicity; and (iii) models for toxicity of psychotropic drugs. The validation of these models has been carried out with several splits into the training, invisible training, calibration, and validation sets. The ability of IIC to be an indicator of predictive potential of QSAR models is confirmed. The updated version of the CORAL software (CORALSEA-2017, http://www.insilico.eu/coral) is available on the Internet.     

Identification of structure-activity relationships for adverse effects of pharmaceuticals in humans: Part B. Use of (Q)SAR systems for early detection of drug-induced hepatobiliary and urinary tract toxicities

This report describes the development of quantitative structure-activity relationship (QSAR) models for predicting rare drug-induced liver and urinary tract injury in humans based upon a database of post-marketing adverse effects (AEs) linked to ∼1600 chemical structures. The models are based upon estimated population exposure using AE proportional reporting ratios. Models were constructed for 5 types of liver injury (liver enzyme disorders, cytotoxic injury, cholestasis and jaundice, bile duct disorders, gall bladder disorders) and 6 types of urinary tract injury (acute renal disorders, nephropathies, bladder disorders, kidney function tests, blood in urine, urolithiases). Identical training data sets were configured for 4 QSAR programs (MC4PC, MDL-QSAR, BioEpisteme, and Predictive Data Miner). Model performance was optimized and was shown to be affected by the AE scoring method and the ratio of the number of active to inactive drugs. The best QSAR models exhibited an overall average 92.4% coverage, 86.5% specificity and 39.3% sensitivity. The 4 QSAR programs were demonstrated to be complementary and enhanced performance was obtained by combining predictions from 2 programs (average 78.4% specificity, 56.2% sensitivity). Consensus predictions resulted in better performance as judged by both internal and external validation experiments.     

Exploiting QSAR models in lead optimization

QSAR models can play a vital role in both the opening phase and the endgame of lead optimization. In the opening phase, there is often a large quantity of data from high-throughput screening (HTS), and potential leads need to be selected from several distinct chemotypes. In the endgame, the throughput of the final, critical ADMET and pharmacokinetic assays is often not sufficient to allow full experimental characterization of all the structures in the available time. A considerable amount of the current research toward new QSAR models is based on the modeling of the general ADMET phenomena, with the aim of constructing globally applicable models. The process to construct QSAR models is relatively straightforward; however, it is also simple to build misleading, or even incorrect, models. This review considers the key developments in the field of QSAR modeling: how QSAR models are constructed, how they can be validated, their reliability and their applicability. If applied carefully and appropriately, the QSAR technique has a valuable role to play during lead optimization.     

Comparative analysis of predictive models for nongenotoxic hepatocarcinogenicity using both toxicogenomics and quantitative structure-activity relationships

The primary testing strategy to identify nongenotoxic carcinogens largely relies on the 2-year rodent bioassay, which is time-consuming and labor-intensive. There is an increasing effort to develop alternative approaches to prioritize the chemicals for, supplement, or even replace the cancer bioassay. In silico approaches based on quantitative structure-activity relationships (QSAR) are rapid and inexpensive and thus have been investigated for such purposes. A slightly more expensive approach based on short-term animal studies with toxicogenomics (TGx) represents another attractive option for this application. Thus, the primary questions are how much better predictive performance using short-term TGx models can be achieved compared to that of QSAR models, and what length of exposure is sufficient for high quality prediction based on TGx. In this study, we developed predictive models for rodent liver carcinogenicity using gene expression data generated from short-term animal models at different time points and QSAR. The study was focused on the prediction of nongenotoxic carcinogenicity since the genotoxic chemicals can be inexpensively removed from further development using various in vitro assays individually or in combination. We identified 62 chemicals whose hepatocarcinogenic potential was available from the National Center for Toxicological Research liver cancer database (NCTRlcdb). The gene expression profiles of liver tissue obtained from rats treated with these chemicals at different time points (1 day, 3 days, and 5 days) are available from the Gene Expression Omnibus (GEO) database. Both TGx and QSAR models were developed on the basis of the same set of chemicals using the same modeling approach, a nearest-centroid method with a minimum redundancy and maximum relevancy-based feature selection with performance assessed using compound-based 5-fold cross-validation. We found that the TGx models outperformed QSAR in every aspect of modeling. For example, the TGx models' predictive accuracy (0.77, 0.77, and 0.82 for the 1-day, 3-day, and 5-day models, respectively) was much higher for an independent validation set than that of a QSAR model (0.55). Permutation tests confirmed the statistical significance of the model's prediction performance. The study concluded that a short-term 5-day TGx animal model holds the potential to predict nongenotoxic hepatocarcinogenicity.