Validation of pesticide root zone model 3.12: employing uncertainty analysis

Computer models are being increasingly used to provide an efficient cost-effective means of evaluating the fate and behavior of chemicals in the environment. These models can be used in lieu of or in conjunction with field studies. Because of the increasing reliance on models for critical regulatory decision making, the need arose to assess the validity of regulatory models via an analysis of the correlation of model response estimates with measured data. In conjunction with the evaluation of the correlation of model response estimates and measured field data, a rigorous statistically based validation was also warranted. Because of the unique nature of the correlative exercise using modeled and measured data, standard statistical analyses, while informative, failed to encompass factors associated with the uncertainty of measured environmental fate data and potential model inputs. In an effort to evaluate this uncertainty, an initial sensitivity analysis was performed where key model input parameters for runoff and leaching simulations were identified. Once the sensitive input parameters were identified, a Monte Carlo-based preprocessor was developed whereby the sampling distributions of these parameters were used to propagate uncertainty in the input parameters into error in model predictions. Importantly, assumptions about parameter distributions for input into the Monte Carlo tool were made only after a formal detailed site-specific analysis of measured field data. Employing the functionality of the Crystal Ball Pro development environment, the pesticide root zone model (PRZM) 3.12 was run iteratively for 500 trials, and model output was collated and analyzed. The model predictions were considered reasonably accurate for most regulatory requirements, and the model prediction error was considered acceptable.     

Comparison of pesticide root zone model 3.12: leaching predictions with field data

As part of a process to improve confidence in the results of regulatory modeling, predictions of the pesticide root zone model (PRZM) 3.12 were compared with measured data collected in nine different field leaching studies. Reasonable estimates of leaching were obtained with PRZM 3.12 in homogeneous soils where preferential flow is not significant. The PRZM 3.12 usually did a good job of predicting movement of bromide in soil (soil and soil pore-water concentrations were generally within a factor of two of predicted values). For simulations based on the best choices for input parameters, predictions of soil pore-water concentrations for pesticides were usually within a factor of three and soil pore-water estimates within a factor of 11. When the model input parameters were calibrated to improve the simulation of hydrology, predicted pesticide concentrations in soil pore water were usually within a factor of two of measured concentrations. Because of the sensitivity of leaching to degradation rate, the most accurate predictions were obtained with pesticides with relatively slow degradation rates. When conservative assumptions were used to define input pesticide parameters, predictions of pesticide concentrations were usually a factor of two greater than when using the best estimate of input parameters without any built-in conservatism.     

Review of the validation of models used in Federal Insecticide,Fungicide, and Rodenticide Act environmental exposure assessments

The first activity of the Federal Insecticide. Fungicide, and Rodenticide Act (FIFRA) Environmental Model Validation Task Force, established to increase confidence in the use of environmental models used in regulatory assessments, was to review the literature information on validation of the pesticide root zone model (PRZM) and the groundwater loading effects of agricultural management systems (GLEAMS). This literature information indicates that these models generally predict the same or greater leaching than observed in actual field measurements, suggesting that these models are suitable for use in regulatory assessments. However, additional validation research conducted using the newest versions of the models would help improve confidence in runoff and leaching predictions because significant revisions have been made in models over the years, few of the literature studies focused on runoff losses, the number of studies having quantitative validation results is minimal, and modelers were aware of the field results in most of the literature studies. Areas for special consideration in conducting model validation research include improving the process for selecting input parameters, developing recommendations for performing calibration simulations, devising appropriate procedures for keeping results of field studies from modelers performing simulations to validate model predictions while providing access for calibration simulations, and developing quantitative statistical procedures for comparing model predictions with experimental results.     

User subjectivity in Monte Carlo modeling of pesticide exposure

Monte Carlo techniques are increasingly used in pesticide exposure modeling to evaluate the uncertainty in predictions arising from uncertainty in input parameters and to estimate the confidence that should be assigned to the modeling results. The approach typically involves running a deterministic model repeatedly for a large number of input values sampled from statistical distributions. In the present study, six modelers made choices regarding the type and parameterization of distributions assigned to degradation and sorption data for an example pesticide, the correlation between the parameters, the tool and method used for sampling, and the number of samples generated. A leaching assessment was carried out using a single model and scenario and all data for sorption and degradation generated by the six modelers. The distributions of sampled parameters differed between the modelers, and the agreement with the measured data was variable. Large differences were found between the upper percentiles of simulated concentrations in leachate. The probability of exceeding 0.1 microg/L ranged from 0 to 35.7%. The present study demonstrated that subjective choices made in Monte Carlo modeling introduce variability into probabilistic modeling and that the results need to be interpreted with care.     

Application of the RICEWQ-VADOFT model for simulating the environmental fate of pretilachlor in rice paddies

No validated models in Europe are capable of simulating the environmental fate of pesticides under the specific conditions of rice fields. Rice water quality--vadose zone flow and transport (RICEWQ-VADOFT) is a model developed from the coupling of a surface runoff model (RICEWQ) and a vadose zone flow and transport model (VADOFT) for determining predicted environmental concentrations in paddy water and sediment, runoff, and groundwater. This study is intended to evaluate the capability of this model to simulate effectively the environmental fate of the herbicide pretilachlor in the paddy environment. A two-year field study conducted in a representative rice-cultivated area of northern Italy provided measured concentrations of pretilachlor in paddy water and sediment and also a limited number of observations on runoff losses. The model successfully predicted the water balance in the paddy field in both years. After limited calibration, the model predicted the fate of pretilachlor in paddy water and sediment with high accuracy. Agreement between predicted and measured concentrations of pretilachlor in both years was assessed statistically using several statistical indicators. For example, modeling efficiency (EF) values of 0.867 to 0.935 and 0.702 to 0.718 in paddy water and sediment, respectively, document the strong agreement between predicted and measured pesticide concentrations. The model predictions showed high agreement with the limited amount of measured runoff data in 2002. The model predicted that no significant amounts of pretilachlor would leach below the top 25 cm of the soil, although no measured data were available to evaluate the predicted results. A sensitivity analysis of the model to variables controlling pesticide partitioning to paddy sediment (VBIND, depth for direct partitioning of pesticide to bed sediment; VMIX, mixing velocity by molecular diffusion) revealed that the predictions of pesticide leaching were influenced strongly by those variables. Generally the RICEWQ-VADOFT model is a useful modeling tool for pesticide risk assessment in rice paddies.     

Comparison of pesticide root zone model 3.12: runoff predictions with field data

As part of a process to improve confidence in the results of regulatory modeling, predictions of pesticide root zone model (PRZM) 3.12 were compared with measured data collected in nine different runoff field studies. This comparison shows that PRZM 3.12 provides a reasonable estimate of chemical runoff at the edge of a field. Simulations based on the best choices for input parameters (no conservatism built into input parameters) are generally within an order of magnitude of measured data, with better agreement observed both for larger events and for cumulative values over the study period. When the model input parameters are calibrated to improve the hydrology, the fit between predicted and observed data improves (results are usually within a factor of three). When conservatism is deliberately introduced into the input pesticide parameters, substantial overprediction of runoff losses occur. Recommendations for future work to improve regulatory models include implementation of more sophisticated evapotranspiration routines, allowing for seasonal variation of various model parameters (such as curve numbers, crop cover, and Manning's surface roughness coefficients), better procedures for estimating site-specific degradation rates in surface and subsoils, and improved sorption routines.     

Using Monte Carlo techniques to judge model prediction accuracy: validation of the pesticide root zone model 3.12

Individuals from the Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA) Environmental Model Validation Task Force (FEMVTF) Statistics Committee periodically met to discuss the mechanism for conducting an uncertainty analysis of Version 3.12 of the pesticide root zone model (PRZM 3.12) and to identify those model input parameters that most contribute to model prediction error. This activity was part of a larger project evaluating PRZM 3.12. The goal of the uncertainty analysis was to compare site-specific model predictions and field measurements using the variability in each as a basis of comparison. Monte Carlo analysis was used as an integral tool for judging the model's ability to predict accurately. The model was judged on how well it predicts measured values, taking into account the uncertainty in the model predictions. Monte Carlo analysis provides the tool for inferring model prediction uncertainty. We argue that this is a fairer test of the model than a simple one-to-one comparison between predictions and measurements. Because models are known to be imperfect predictors prior to running the model, the inaccuracy in model predictions should be considered when models are judged for their predictive ability. Otherwise, complex models can easily fail a validation test. Few complex models, such as PRZM 3.12, would pass a typical model validation exercise. This paper describes the approaches to the validation of PRZM 3.12 used by the committee and discusses issues in sampling distribution selection and appropriate statistics for interpreting the model validation results.     

Issues of replicability in Monte Carlo modeling: a case study with a pesticide leaching model

Sensitivity and uncertainty analyses based on Monte Carlo sampling were undertaken for various numbers of runs of the pesticide leaching model (PELMO). Analyses were repeated 10 times with different seed numbers. The ranking of PELMO input parameters according to their influence on predictions for leaching was stable for the most influential parameters. For less influential parameters, the sensitivity ranking was severely influenced by the seed number used. For uncertainty analyses, probabilities of exceeding a particular concentration were significantly influenced by the seed number used in the random sampling of values for the two parameters considered, even for those cases in which 5,000 model runs were undertaken (coefficient of variation of 10 replicated analyses, 5%). A decrease in the variability of exceedance probabilities could be achieved by further increasing the number of model runs. However, this may prove to be impractical when complex deterministic models with a relatively long running time are used. Attention should be paid to replicability aspects by modelers when devising their approach to assessing the uncertainty associated with the modeling and by decision makers when examining the results of probabilistic approaches.     

Modeled estimates of chlorpyrifos exposure and dose for the Minnesota and Arizona NHEXAS populations

This paper presents a probabilistic, multimedia, multipathway exposure model and assessment for chlorpyrifos developed as part of the National Human Exposure Assessment Survey (NHEXAS). The model was constructed using available information prior to completion of the NHEXAS study. It simulates the distribution of daily aggregate and pathway-specific chlorpyrifos absorbed dose in the general population of the State of Arizona (AZ) and in children aged 3-12 years residing in Minneapolis-St. Paul, Minnesota (MSP). Pathways included were inhalation of indoor and outdoor air, dietary ingestion, non-dietary ingestion of dust and soil, and dermal contact with dust and soil. Probability distributions for model input parameters were derived from the available literature, and input values were chosen to represent chlorpyrifos concentrations and demographics in AZ and MSP to the extent possible. When the NHEXAS AZ and MSP data become available, they can be compared to the distributions derived in this and other prototype modeling assessments to test the adequacy of this pre-NHEXAS model assessment. Although pathway-specific absorbed dose estimates differed between AZ and MSP due to differences in model inputs between simulated adults and children, the aggregate model results and general findings for simulated AZ and MSP populations were similar. The major route of chlorpyrifos intake was food ingestion, followed by indoor air inhalation. Two-stage Monte Carlo simulation was used to derive estimates of both inter-individual variability and uncertainty in the estimated distributions. The variability in the model results reflects the difference in activity patterns, exposure factors, and concentrations contacted by individuals during their daily activities. Based on the coefficient of variation, indoor air inhalation and dust ingestion were most variable relative to the mean, primarily because of variability in concentrations due to use or no-use of pesticides. Uncertainty analyses indicated a factor of 10-30 for uncertainty of model predictions of 10th, 50th, and 90th percentiles. The greatest source of uncertainty in the model stems from the definition of no household pesticide use as no use in the past year. Because chlorpyrifos persists in the residential environment for longer than a year, the modeled estimates are likely to be low. More information on pesticide usage and environmental concentrations measured at different post-application times is needed to refine and evaluate this and other pesticide exposure models.     

Ensemble learning of inverse probability weights for marginal structural modeling in large observational datasets

Inverse probability weights used to fit marginal structural models are typically estimated using logistic regression. However, a data‐adaptive procedure may be able to better exploit information available in measured covariates. By combining predictions from multiple algorithms, ensemble learning offers an alternative to logistic regression modeling to further reduce bias in estimated marginal structural model parameters. We describe the application of two ensemble learning approaches to estimating stabilized weights: super learning (SL), an ensemble machine learning approach that relies on V‐fold cross validation, and an ensemble learner (EL) that creates a single partition of the data into training and validation sets. Longitudinal data from two multicenter cohort studies in Spain (CoRIS and CoRIS‐MD) were analyzed to estimate the mortality hazard ratio for initiation versus no initiation of combined antiretroviral therapy among HIV positive subjects. Both ensemble approaches produced hazard ratio estimates further away from the null, and with tighter confidence intervals, than logistic regression modeling. Computation time for EL was less than half that of SL. We conclude that ensemble learning using a library of diverse candidate algorithms offers an alternative to parametric modeling of inverse probability weights when fitting marginal structural models. With large datasets, EL provides a rich search over the solution space in less time than SL with comparable results. Copyright © 2014 John Wiley & Sons, Ltd.     

Evaluating and expressing the propagation of uncertainty in chemical fate and bioaccumulation models

First-order analytical sensitivity and uncertainty analysis for environmental chemical fate models is described and applied to a regional contaminant fate model and a food web bioaccumulation model. By assuming linear relationships between inputs and outputs, independence, and log-normal distributions of input variables, a relationship between uncertainty in input parameters and uncertainty in output parameters can be derived, yielding results that are consistent with a Monte Carlo analysis with similar input assumptions. A graphical technique is devised for interpreting and communicating uncertainty propagation as a function of variance in input parameters and model sensitivity. The suggested approach is less calculationally intensive than Monte Carlo analysis and is appropriate for preliminary assessment of uncertainty when models are applied to generic environments or to large geographic areas or when detailed parameterization of input uncertainties is unwarranted or impossible. This approach is particularly useful as a starting point for identification of sensitive model inputs at the early stages of applying a generic contaminant fate model to a specific environmental scenario, as a tool to support refinements of the model and the uncertainty analysis for site-specific scenarios, or for examining defined end points. The analysis identifies those input parameters that contribute significantly to uncertainty in outputs, enabling attention to be focused on defining median values and more appropriate distributions to describe these variables.     

Integrated ecological risk assessment of pesticides in tropical ecosystems: a case study with carbofuran in Brazil

The aim of the present study is to contribute an ecologically relevant assessment of the ecotoxicological effects of pesticide applications in agricultural areas in the tropics, using an integrated approach with information gathered from soil and aquatic compartments. Carbofuran, an insecticide/nematicide used widely on sugarcane crops, was selected as a model substance. To evaluate the toxic effects of pesticide spraying for soil biota, as well as the potential indirect effects on aquatic biota resulting from surface runoff and/or leaching, field and laboratory (using a cost-effective simulator of pesticide applications) trials were performed. Standard ecotoxicological tests were performed with soil (Eisenia andrei, Folsomia candida, and Enchytraeus crypticus) and aquatic (Ceriodaphnia silvestrii) organisms, using serial dilutions of soil, eluate, leachate, and runoff samples. Among soil organisms, sensitivity was found to be E. crypticus < E. andrei < F. candida. Among the aqueous extracts, mortality of C. silvestrii was extreme in runoff samples, whereas eluates were by far the least toxic samples. A generally higher toxicity was found in the bioassays performed with samples from the field trial, indicating the need for improvements in the laboratory simulator. However, the tool developed proved to be valuable in evaluating the toxic effects of pesticide spraying in soils and the potential risks for aquatic compartments.     

A model for evaluating intervention strategies to control salmonella in the poultry meat production chain

A model of the transmission of salmonella through the poultry meat production chain is developed, to predict the effects of intervention strategies for salmonella control. The model first describes the situation before intervention in terms of salmonella prevalences at flock level and some transmission parameters. After single control measures are translated into effects on these transmission parameters, the effects of sets of control measures (intervention strategies), can be calculated with the model. As research data are lacking, the model input parameters were derived from expert opinion. As an example, the effects of two intervention strategies proposed for the Dutch poultry industry are predicted. A sensitivity analysis is performed to indicate where the most effective control measures may be expected. Additionally, the reliability of the model predictions is studied by an uncertainty analysis. The use of the model as a tool for policy makers deciding about salmonella control strategies is discussed.     

Reliability of a new biokinetic model of zirconium in internal dosimetry: part I, parameter uncertainty analysis

The reliability of biokinetic models is essential in internal dose assessments and radiation risk analysis for the public, occupational workers, and patients exposed to radionuclides. In this paper, a method for assessing the reliability of biokinetic models by means of uncertainty and sensitivity analysis was developed. The paper is divided into two parts. In the first part of the study published here, the uncertainty sources of the model parameters for zirconium (Zr), developed by the International Commission on Radiological Protection (ICRP), were identified and analyzed. Furthermore, the uncertainty of the biokinetic experimental measurement performed at the Helmholtz Zentrum München-German Research Center for Environmental Health (HMGU) for developing a new biokinetic model of Zr was analyzed according to the Guide to the Expression of Uncertainty in Measurement, published by the International Organization for Standardization. The confidence interval and distribution of model parameters of the ICRP and HMGU Zr biokinetic models were evaluated. As a result of computer biokinetic modelings, the mean, standard uncertainty, and confidence interval of model prediction calculated based on the model parameter uncertainty were presented and compared to the plasma clearance and urinary excretion measured after intravenous administration. It was shown that for the most important compartment, the plasma, the uncertainty evaluated for the HMGU model was much smaller than that for the ICRP model; that phenomenon was observed for other organs and tissues as well. The uncertainty of the integral of the radioactivity of Zr up to 50 y calculated by the HMGU model after ingestion by adult members of the public was shown to be smaller by a factor of two than that of the ICRP model. It was also shown that the distribution type of the model parameter strongly influences the model prediction, and the correlation of the model input parameters affects the model prediction to a certain extent depending on the strength of the correlation. In the case of model prediction, the qualitative comparison of the model predictions with the measured plasma and urinary data showed the HMGU model to be more reliable than the ICRP model; quantitatively, the uncertainty model prediction by the HMGU systemic biokinetic model is smaller than that of the ICRP model. The uncertainty information on the model parameters analyzed in this study was used in the second part of the paper regarding a sensitivity analysis of the Zr biokinetic models.     

A model for azinphosmethyl attenuation and movement in a Michigan orchard ecosystem: II. Parameterization of a field-based model

Residue data from field samples were used to parameterize a model for azinphosmethyl attenuation and movement in an orchard ecosystem. Rates of attenuation within, and movement between, specified orchard compartments were determined under various rainfall regimes. The output of this model was structured to allow the estimation of the time course of azinphosmethyl exposure to ground-dwelling invertebrates. Root mean squared errors for the comparison of the model predictions with an independent set of residue data indicated good prediction of azinphosmethyl fate within the tree, grass-broadleaves layers, and soil layers. Prediction of pesticide dynamics within the litter-moss was much more difficult. Model predictions estimate that under dry conditions 25% of the daily loss of azinphosmethyl from the orchard trees is due to movement to other parts of the orchard. Greater movement is predicted under rainfall conditions.     

基因表达对膀胱癌生存结局的预测:稀疏与混合Cox模型的实证比较研究

目的 研究稀疏Cox(coxlasso)与混合Cox模型(coxlmm)在全基因表达数据中对膀胱癌预后的预测表现.方法 通过计算一致性指数(C-index)评价两种模型在膀胱癌全基因表达数据中(TCGA,GSE31684和GSE13507)的预测精度,同时在混合Cox模型中将膀胱癌的生存方差划分为临床(PCE)和转录组...     

Examination of the uncertainty in air concentration predictions using Hanford field data

The accuracy of an environmental transport model is best determined by comparing model predictions with environmental measurements made under conditions similar to those assumed by the model, a process commonly referred to as model validation. Over the past several years, we have done a variety of validation studies with the popular Gaussian plume atmospheric dispersion model using data from tests conducted on the Hanford site. Data for short-term releases of small particles for a range of release heights from surface level to 111 m have been used. Downwind distances examined have ranged from a few hundred meters to 12.8 km, depending on the particular data used. Measured and predicted ground-level centerline, crosswind-integrated, and 22.5 degrees sector-averaged air concentrations have been compared. Up to six different sets of atmospheric dispersion parameters and three different atmospheric stability class specification schemes have been examined. Overall, dispersion parameters based on measurements made near Jülich, Federal Republic of Germany, give the best comparisons between observed and predicted air concentrations. The commonly used vertical temperature gradient method for determining atmospheric stability class consistently gives poor results. The accuracy of air concentration predictions improves when dry deposition processes are included in the model. Further validation studies using various Hanford data sets are planned.     

Measurement and Computation of Movement of Bromide Ions and Carbofuran in Ridged Humic-Sandy Soil

Water flow and pesticide transport in the soil of fields with ridges and furrows may be more complex than in the soil of more level fields. Prior to crop emergence, the tracer bromide ion and the insecticide carbofuran were sprayed on the humic-sandy soil of a potato field with ridges and furrows. Rainfall was supplemented by sprinkler irrigation. The distribution of the substances in the soil profile of the ridges and furrows was measured on three dates in the potato growing season. Separate ridge and furrow systems were simulated by using the pesticide emission assessment at regional and local scales (PEARL) model for pesticide behavior in soil–plant systems. The substances travelled deeper in the furrow soil than in the ridge soil, because of runoff from the ridges to the furrows. At 19 days after application, the peak of the bromide distribution was measured to be in the 0.1–0.2 m layer of the ridges, while it was in the 0.3–0.5 m layer of the furrows. After 65 days, the peak of the carbofuran distribution in the ridge soil was still in the 0.1 m top layer, while the pesticide was rather evenly distributed in the top 0.6 m of the furrow soil. The wide ranges in concentration measured with depth showed that preferential water flow and substance transport occurred in the sandy soil. Part of the bromide ion distribution was measured to move faster in soil than the computed wave. The runoff of water and pesticide from the ridges to the furrows, and the thinner root zone in the furrows, are expected to increase the risk of leaching to groundwater in ridged fields, in comparison with more level fields.