Nonlinear analog predictor analysis: a coupled neural network/analog model for climate downscaling.

Synoptic downscaling models are used in climatology to predict values of weather elements at one or more stations based on values of synoptic-scale atmospheric circulation variables. This paper presents a hybrid method for climate prediction and downscaling that couples an analog, i.e., k-nearest neighbor, model to an artificial neural network (ANN) model. In the proposed method, which is based on nonlinear principal predictor analysis (NLPPA), the analog model is embedded inside an ANN, forming its output layer. Nonlinear analog predictor analysis (NLAPA) is a flexible model that maintains the ability of the analog model to preserve inter-variable relationships and model non-normal and conditional variables (such as precipitation), while taking advantage of NLPPA's ability to define an optimal set of analog predictors that maximize predictive performance. Performance on both synthetic and real-world hydroclimatological benchmark tasks indicates that the NLAPA model is capable of outperforming other forms of analog models commonly used in synoptic downscaling.     

Schizophrenic Patients and Their Unaffected Siblings Share Increased Resting-State Connectivity in the Task-Negative Network but Not Its Anticorrelated Task-Positive Network

Background:

Abnormal connectivity of the anticorrelated intrinsic networks, the task-negative network (TNN), and the task-positive network (TPN) is implicated in schizophrenia. Comparisons between schizophrenic patients and their unaffected siblings enable further understanding of illness susceptibility and pathophysiology. We examined the resting-state connectivity differences in the intrinsic networks between schizophrenic patients, their unaffected siblings, and healthy controls.

Methods:

Resting-state functional magnetic resonance images were obtained from 25 individuals in each subject group. The posterior cingulate cortex/precuneus and right dorsolateral prefrontal cortex were used as seed regions to identify the TNN and TPN through functional connectivity analysis. Interregional connectivity strengths were analyzed using overlapped intrinsic networks composed of regions common to all subject groups.

Results:

Schizophrenic patients and their unaffected siblings showed increased connectivity in the TNN between the bilateral inferior temporal gyri. By contrast, schizophrenic patients alone demonstrated increased connectivity between the posterior cingulate cortex/precuneus and left inferior temporal gyrus and between the ventral medial prefrontal cortex and right lateral parietal cortex in the TNN. Schizophrenic patients exhibited increased connectivity between the left dorsolateral prefrontal cortex and right inferior frontal gyrus in the TPN relative to their unaffected siblings, though this trend only approached statistical significance in comparison to healthy controls.

Conclusion:

Resting-state hyperconnectivity of the intrinsic networks may disrupt network coordination and thereby contribute to the pathophysiology of schizophrenia. Similar, though milder, hyperconnectivity of the TNN in unaffected siblings of schizophrenic patients may contribute to the identification of schizophrenia endophenotypes and ultimately to the determination of schizophrenia risk genes.     

Exploratory analysis of climate data using source separation methods.

We present an example of exploratory data analysis of climate measurements using a recently developed denoising source separation (DSS) framework. We analyzed a combined dataset containing daily measurements of three variables: surface temperature, sea level pressure and precipitation around the globe, for a period of 56 years. Components exhibiting slow temporal behavior were extracted using DSS with linear denoising. The first component, most prominent in the interannual time scale, captured the well-known El Niño-Southern Oscillation (ENSO) phenomenon and the second component was close to the derivative of the first one. The slow components extracted in a wider frequency range were further rotated using a frequency-based separation criterion implemented by DSS with nonlinear denoising. The rotated sources give a meaningful representation of the slow climate variability as a combination of trends, interannual oscillations, the annual cycle and slowly changing seasonal variations. Again, components related to the ENSO phenomenon emerge very clearly among the found sources.     

Predictive uncertainty in environmental modelling.

Artificial neural networks have proved an attractive approach to non-linear regression problems arising in environmental modelling, such as statistical downscaling, short-term forecasting of atmospheric pollutant concentrations and rainfall run-off modelling. However, environmental datasets are frequently very noisy and characterized by a noise process that may be heteroscedastic (having input dependent variance) and/or non-Gaussian. The aim of this paper is to review existing methodologies for estimating predictive uncertainty in such situations and, more importantly, to illustrate how a model of the predictive distribution may be exploited in assessing the possible impacts of climate change and to improve current decision making processes. The results of the WCCI-2006 predictive uncertainty in environmental modelling challenge are also reviewed, suggesting a number of areas where further research may provide significant benefits.     

An information theoretic approach for combining neural network process models

Typically neural network modelers in chemical engineering focus on identifying and using a single, hopefully optimal, neural network model. Using a single optimal model implicitly assumes that one neural network model can extract all the information available in a given data set and that the other candidate models are redundant. In general, there is no assurance that any individual model has extracted all relevant information from the data set. Recently, Wolpert (Neural Networks, 5(2), 241 (1992)) proposed the idea of stacked generalization to combine multiple models. Sridhar, Seagrave and Barlett (AIChE J., 42, 2529 (1996)) implemented the stacked generalization for neural network models by integrating multiple neural networks into an architecture known as stacked neural networks (SNNs). SNNs consist of a combination of the candidate neural networks and were shown to provide improved modeling of chemical processes. However, in Sridhar's work SNNs were limited to using a linear combination of artificial neural networks. While a linear combination is simple and easy to use, it can utilize only those model outputs that have a high linear correlation to the output. Models that are useful in a nonlinear sense are wasted if a linear combination is used. In this work we propose an information theoretic stacking (ITS) algorithm for combining neural network models. The ITS algorithm identifies and combines useful models regardless of the nature of their relationship to the actual output. The power of the ITS algorithm is demonstrated through three examples including application to a dynamic process modeling problem. The results obtained demonstrate that the SNNs developed using the ITS algorithm can achieve highly improved performance as compared to selecting and using a single hopefully optimal network or using SNNs based on a linear combination of neural networks.     

Neural network approach for robust and fast calculation of physical processes in numerical environmental models: Compound parameterization with a quality control of larger errors

Development of neural network (NN) emulations for fast calculations of physical processes in numerical climate and weather prediction models depends significantly on our ability to generate a representative training set. Owing to the high dimensionality of the NN input vector which is of the order of several hundreds or more, it is rather difficult to cover the entire domain, especially its “far corners” associated with rare events, even when we use model simulated data for the NN training. Moreover the domain may evolve (e.g., due to climate change). In this situation the emulating NN may be forced to extrapolate beyond its generalization ability and may lead to larger errors in NN outputs. A new technique, a compound parameterization, has been developed to address this problem and to make the NN emulation approach more suitable for long-term climate prediction and climate change projections and other numerical modeling applications. Two different designs of the compound parameterization are presented and discussed.     

Trial-to-trial variability of cortical evoked responses: implications for the analysis of functional connectivity.

OBJECTIVES: The time series of single trial cortical evoked potentials typically have a random appearance, and their trial-to-trial variability is commonly explained by a model in which random ongoing background noise activity is linearly combined with a stereotyped evoked response. In this paper, we demonstrate that more realistic models, incorporating amplitude and latency variability of the evoked response itself, can explain statistical properties of cortical potentials that have often been attributed to stimulus-related changes in functional connectivity or other intrinsic neural parameters. METHODS: Implications of trial-to-trial evoked potential variability for variance, power spectrum, and interdependence measures like cross-correlation and spectral coherence, are first derived analytically. These implications are then illustrated using model simulations and verified experimentally by the analysis of intracortical local field potentials recorded from monkeys performing a visual pattern discrimination task. To further investigate the effects of trial-to-trial variability on the aforementioned statistical measures, a Bayesian inference technique is used to separate single-trial evoked responses from the ongoing background activity. RESULTS: We show that, when the average event-related potential (AERP) is subtracted from single-trial local field potential time series, a stimulus phase-locked component remains in the residual time series, in stark contrast to the assumption of the common model that no such phase-locked component should exist. Two main consequences of this observation are demonstrated for statistical measures that are computed on the residual time series. First, even though the AERP has been subtracted, the power spectral density, computed as a function of time with a short sliding window, can nonetheless show signs of modulation by the AERP waveform. Second, if the residual time series of two channels co-vary, then their cross-correlation and spectral coherence time functions can also be modulated according to the shape of the AERP waveform. Bayesian estimation of single-trial evoked responses provides further proof that these time-dependent statistical changes are due to remnants of the evoked phase-locked component in the residual time series. CONCLUSIONS: Because trial-to-trial variability of the evoked response is commonly ignored as a contributing factor in evoked potential studies, stimulus-related modulations of power spectral density, cross-correlation, and spectral coherence measures is often attributed to dynamic changes of the connectivity within and among neural populations. This work demonstrates that trial-to-trial variability of the evoked response must be considered as a possible explanation of such modulation.     

Clustering ensembles of neural network models.

We show that large ensembles of (neural network) models, obtained e.g. in bootstrapping or sampling from (Bayesian) probability distributions, can be effectively summarized by a relatively small number of representative models. In some cases this summary may even yield better function estimates. We present a method to find representative models through clustering based on the models' outputs on a data set. We apply the method on an ensemble of neural network models obtained from bootstrapping on the Boston housing data, and use the results to discuss bootstrapping in terms of bias and variance. A parallel application is the prediction of newspaper sales, where we learn a series of parallel tasks. The results indicate that it is not necessary to store all samples in the ensembles: a small number of representative models generally matches, or even surpasses, the performance of the full ensemble. The clustered representation of the ensemble obtained thus is much better suitable for qualitative analysis, and will be shown to yield new insights into the data.     

Cyclical konzo epidemics and climate variability

Konzo epidemics have occurred during droughts in the Democratic Republic of Congo (DR Congo) for >70 years, but also in Mozambique, Tanzania, and the Central African Republic. The illness is attributed to exposure to cyanide from cassava foods, on which the population depends almost exclusively during droughts. Production of cassava, a drought‐resistant crop, has been shown to correlate with cyclical changes in precipitation in konzo‐affected countries. Here we review the epidemiology of konzo as well as models of its pathogenesis. A spectral analysis of precipitation and konzo is performed to determine whether konzo epidemics are cyclical and whether there is spectral coherence. Time series of environmental temperature, precipitation, and konzo show cyclical changes. Periodicities of dominant frequencies in the spectra of precipitation and konzo range from 3 to 6 years in DR Congo. There is coherence of the spectra of precipitation and konzo. The magnitude squared coherence of 0.9 indicates a strong relationship between variability of climate and konzo epidemics. Thus, it appears that low precipitation phases of climate variability reduce the yield of food crops except cassava, upon which the population depends for supply of calories during droughts. Presence of very high concentrations of thiocyanate (SCN^?), the major metabolite of cyanide, in the bodily fluids of konzo subjects is a consequence of dietary exposure to cyanide, which follows intake of poorly processed cassava roots. Because cyanogens and minor metabolites of cyanide have not induced konzo‐like illnesses, SCN^? remains the most likely neurotoxicant of konzo. Public health control of konzo will require food and water programs during droughts. [Correction added on 26 February 2015, after first online publication: abstract reformatted per journal style] Ann Neurol 2015;77:371–380     

Network anticorrelations,global regression,and phase‐shifted soft tissue correction

Synchronized low‐frequency BOLD fluctuations are observed in dissociable large‐scale, distributed networks with functional specialization. Two such networks, referred to as the task‐positive network (TPN) and the task‐negative network (TNN) because they tend to be active or inactive during cognitively demanding tasks, show reproducible anticorrelation of resting BOLD fluctuations after removal of the global brain signal. Because global signal regression mandates that anticorrelated regions to a given seed region must exist, it is unclear whether such anticorrelations are an artifact of global regression or an intrinsic property of neural activity. In this study, we demonstrate from simulated data that spurious anticorrelations are introduced during global regression for any two networks as a linear function of their size. Using actual resting state data, we also show that both the TPN and TNN become anticorrelated with the orbits when soft tissues are included in the global regression algorithm. Finally, we propose a technique using phase‐shifted soft tissue regression (PSTCor) that allows improved correction of global physiological artifacts without global regression that shows improved anatomic specificity to global regression but does not show significant network anticorrelations. These results imply that observed anticorrelations between TNN and TPN may be largely or entirely artifactual in the resting state. These results also imply that differences in network anticorrelations attributed to pathophysiological or behavioral states may be due to differences in network size or recruitment rather than actual anticorrelations. Hum Brain Mapp, 2011. © 2010 Wiley‐Liss, Inc.     

Synthetic event-related potentials: A computational bridge between neurolinguistic models and experiments

Our previous work developed Synthetic Brain Imaging to link neural and schema network models of cognition and behavior to PET and fMRI studies of brain function. We here extend this approach to Synthetic Event-Related Potentials (Synthetic ERP). Although the method is of general applicability, we focus on ERP correlates of language processing in the human brain. The method has two components: Phase 1: To generate cortical electro-magnetic source activity from neural or schema network models; and Phase 2: To generate known neurolinguistic ERP data (ERP scalp voltage topographies and waveforms) from putative cortical source distributions and activities within a realistic anatomical model of the human brain and head. To illustrate the challenges of Phase 2 of the methodology, spatiotemporal information from Friederici’s 2002 model of auditory language comprehension was used to define cortical regions and time courses of activation for implementation within a forward model of ERP data. The cortical regions from the 2002 model were modeled using atlas-based masks overlaid on the MNI high definition single subject cortical mesh. The electromagnetic contribution of each region was modeled using current dipoles whose position and orientation were constrained by the cortical geometry. In linking neural network computation via EEG forward modeling to empirical results in neurolinguistics, we emphasize the need for neural network models to link their architecture to geometrically sound models of the cortical surface, and the need for conceptual models to refine and adopt brain-atlas based approaches to allow precise brain anchoring of their modules. The detailed analysis of Phase 2 sets the stage for a brief introduction to Phase 1 of the program, including the case for a schema-theoretic approach to language production and perception presented in detail elsewhere. Unlike Dynamic Causal Modeling (DCM) and Bojak’s mean field model, Synthetic ERP builds on models of networks that mediate the relation between the brain’s inputs, outputs, and internal states in executing a specific task. The neural networks used for Synthetic ERP must include neuroanatomically realistic placement and orientation of the cortical pyramidal neurons. These constraints pose exciting challenges for future work in neural network modeling that is applicable to systems and cognitive neuroscience.     

Neural timing nets.

Formulations of artificial neural networks are directly related to assumptions about neural coding in the brain. Traditional connectionist networks assume channel-based rate coding, while time-delay networks convert temporally-coded inputs into rate-coded outputs. Neural timing nets that operate on time structured input spike trains to produce meaningful time-structured outputs are proposed. Basic computational properties of simple feedforward and recurrent timing nets are outlined and applied to auditory computations. Feed-forward timing nets consist of arrays of coincidence detectors connected via tapped delay lines. These temporal sieves extract common spike patterns in their inputs that can subserve extraction of common fundamental frequencies (periodicity pitch) and common spectrum (timbre). Feedforward timing nets can also be used to separate time-shifted patterns, fusing patterns with similar internal temporal structure and spatially segregating different ones. Simple recurrent timing nets consisting of arrays of delay loops amplify and separate recurring time patterns. Single- and multichannel recurrent timing nets are presented that demonstrate the separation of concurrent, double vowels. Timing nets constitute a new and general neural network strategy for performing temporal computations on neural spike trains: extraction of common periodicities, detection of recurring temporal patterns, and formation and separation of invariant spike patterns that subserve auditory objects.     

Support for an auto-associative model of spoken cued recall: evidence from fMRI

Cued recall and item recognition are considered the standard episodic memory retrieval tasks. However, only the neural correlates of the latter have been studied in detail with fMRI. Using an event-related fMRI experimental design that permits spoken responses, we tested hypotheses from an auto-associative model of cued recall and item recognition [Chappell, M., & Humphreys, M. S. (1994). An auto-associative neural network for sparse representations: Analysis and application to models of recognition and cued recall. Psychological Review, 101, 103-128]. In brief, the model assumes that cues elicit a network of phonological short term memory (STM) and semantic long term memory (LTM) representations distributed throughout the neocortex as patterns of sparse activations. This information is transferred to the hippocampus which converges upon the item closest to a stored pattern and outputs a response. Word pairs were learned from a study list, with one member of the pair serving as the cue at test. Unstudied words were also intermingled at test in order to provide an analogue of yes/no recognition tasks. Compared to incorrectly rejected studied items (misses) and correctly rejected (CR) unstudied items, correctly recalled items (hits) elicited increased responses in the left hippocampus and neocortical regions including the left inferior prefrontal cortex (LIPC), left mid lateral temporal cortex and inferior parietal cortex, consistent with predictions from the model. This network was very similar to that observed in yes/no recognition studies, supporting proposals that cued recall and item recognition involve common rather than separate mechanisms.     

Statistical tools to assess the reliability of self-organizing maps.

Results of neural network learning are always subject to some variability, due to the sensitivity to initial conditions, to convergence to local minima, and, sometimes more dramatically, to sampling variability. This paper presents a set of tools designed to assess the reliability of the results of self-organizing maps (SOM), i.e. to test on a statistical basis the confidence we can have on the result of a specific SOM. The tools concern the quantization error in a SOM, and the neighborhood relations (both at the level of a specific pair of observations and globally on the map). As a by-product, these measures also allow to assess the adequacy of the number of units chosen in a map. The tools may also be used to measure objectively how the SOM are less sensitive to non-linear optimization problems (local minima, convergence, etc.) than other neural network models.     

Wavelet neural network classification of EEG signals by using AR model with MLE preprocessing.

Since EEG is one of the most important sources of information in therapy of epilepsy, several researchers tried to address the issue of decision support for such a data. In this paper, we introduce two fundamentally different approaches for designing classification models (classifiers); the traditional statistical method based on logistic regression and the emerging computationally powerful techniques based on artificial neural networks (ANNs). Logistic regression as well as feedforward error backpropagation artificial neural networks (FEBANN) and wavelet neural networks (WNN) based classifiers were developed and compared in relation to their accuracy in classification of EEG signals. In these methods we used FFT and autoregressive (AR) model by using maximum likelihood estimation (MLE) of EEG signals as an input to classification system with two discrete outputs: epileptic seizure or nonepileptic seizure. By identifying features in the signal we want to provide an automatic system that will support a physician in the diagnosing process. By applying AR with MLE in connection with WNN, we obtained novel and reliable classifier architecture. The network is constructed by the error backpropagation neural network using Morlet mother wavelet basic function as node activation function. The comparisons between the developed classifiers were primarily based on analysis of the receiver operating characteristic (ROC) curves as well as a number of scalar performance measures pertaining to the classification. The WNN-based classifier outperformed the FEBANN and logistic regression based counterpart. Within the same group, the WNN-based classifier was more accurate than the FEBANN-based classifier, and the logistic regression-based classifier.     

Model selection in neural networks

In this article, we examine how model selection in neural networks can be guided by statistical procedures such as hypothesis tests, information criteria and cross validation. The application of these methods in neural network models is discussed, paying attention especially to the identification problems encountered. We then propose five specification strategies based on different statistical procedures and compare them in a simulation study. As the results of the study are promising, it is suggested that a statistical analysis should become an integral part of neural network modeling.     

Trainable fusion rules. I. Large sample size case

A wide selection of standard statistical pattern classification algorithms can be applied as trainable fusion rules while designing neural network ensembles. A focus of the present two-part paper is finite sample effects: the complexity of base classifiers and fusion rules; the type of outputs provided by experts to the fusion rule; non-linearity of the fusion rule; degradation of experts and the fusion rule due to the lack of information in the design set; the adaptation of base classifiers to training set size, etc. In the first part of this paper, we consider arguments for utilizing continuous outputs of base classifiers versus categorical outputs and conclude: if one succeeds in having a small number of expert networks working perfectly in different parts of the input feature space, then crisp outputs may be preferable over continuous outputs. Afterwards, we oppose fixed fusion rules versus trainable ones and demonstrate situations where weighted average fusion can outperform simple average fusion. We present a review of statistical classification rules, paying special attention to these linear and non-linear rules, which are employed rarely but, according to our opinion, could be useful in neural network ensembles. We consider ideal and sample-based oracle decision rules and illustrate characteristic features of diverse fusion rules by considering an artificial two-dimensional (2D) example where the base classifiers perform well in different regions of input feature space.     

A Novel Model Incorporating Two Variability Sources for Describing Motor Evoked Potentials

ObjectiveMotor evoked potentials (MEPs) play a pivotal role in transcranial magnetic stimulation (TMS), e.g., for determining the motor threshold and probing cortical excitability. Sampled across the range of stimulation strengths, MEPs outline an input–output (IO) curve, which is often used to characterize the corticospinal tract. More detailed understanding of the signal generation and variability of MEPs would provide insight into the underlying physiology and aid correct statistical treatment of MEP data.MethodsA novel regression model is tested using measured IO data of twelve subjects. The model splits MEP variability into two independent contributions, acting on both sides of a strong sigmoidal nonlinearity that represents neural recruitment. Traditional sigmoidal regression with a single variability source after the nonlinearity is used for comparison.ResultsThe distribution of MEP amplitudes varied across different stimulation strengths, violating statistical assumptions in traditional regression models. In contrast to the conventional regression model, the dual variability source model better described the IO characteristics including phenomena such as changing distribution spread and skewness along the IO curve.ConclusionsMEP variability is best described by two sources that most likely separate variability in the initial excitation process from effects occurring later on. The new model enables more accurate and sensitive estimation of the IO curve characteristics, enhancing its power as a detection tool, and may apply to other brain stimulation modalities. Furthermore, it extracts new information from the IO data concerning the neural variability—information that has previously been treated as noise.     

A network approach to climate change anxiety and its key related features

Research has pointed to startling worldwide rates of people reporting considerable anxiety vis-à-vis climate change. Yet, uncertainties remain regarding how climate anxiety’s cognitive-emotional features and daily life functional impairments interact with one another and with climate change experience, pro-environmental behaviors, and general worry. In this study, we apply network analyses to examine the associations among these variables in an international community sample (n = 874). We computed two network models, a graphical Gaussian model to explore network structure, potential communities, and influential nodes, and a directed acyclic graph to examine the probabilistic dependencies among the variables. Both network models pointed to the cognitive-emotional features of climate anxiety as a potential hub bridging general worry, the experience of climate change, pro-environmental behaviors, and the functional impairments associated with climate anxiety. Our findings offer data‐driven clues for the field’s larger quest to establish the foundations of climate anxiety.     

Abnormal dynamic functional connectivity after sleep deprivation from temporal variability perspective

Sleep deprivation (SD) is very common in modern society and regarded as a potential causal mechanism of several clinical disorders. Previous neuroimaging studies have explored the neural mechanisms of SD using magnetic resonance imaging (MRI) from static (comparing two MRI sessions [one after SD and one after resting wakefulness]) and dynamic (using repeated MRI during one night of SD) perspectives. Recent SD researches have focused on the dynamic functional brain organization during the resting‐state scan. Our present study adopted a novel metric (temporal variability), which has been successfully applied to many clinical diseases, to examine the dynamic functional connectivity after SD in 55 normal young subjects. We found that sleep‐deprived subjects showed increased regional‐level temporal variability in large‐scale brain regions, and decreased regional‐level temporal variability in several thalamus subregions. After SD, participants exhibited enhanced intra‐network temporal variability in the default mode network (DMN) and increased inter‐network temporal variability in numerous subnetwork pairs. Furthermore, we found that the inter‐network temporal variability between visual network and DMN was negative related with the slowest 10% respond speed (β = −.42, p = 5.57 × 10⁻⁴) of the psychomotor vigilance test after SD following the stepwise regression analysis. In conclusion, our findings suggested that sleep‐deprived subjects showed abnormal dynamic brain functional configuration, which provides new insights into the neural underpinnings of SD and contributes to our understanding of the pathophysiology of clinical disorders.