Learning From Surprise: Harnessing a Metacognitive Surprise Signal to Build and Adapt Belief Networks

One's level of surprise can be thought of as a metacognitive signal indicating how well one can explain new information. We discuss literature on how this signal can be used adaptively to build, and, when necessary, reorganize belief networks. We present challenges in the use of a surprise signal, such as hindsight bias and the tendency to equate difficulty with implausibility, and point to evidence suggesting that one can overcome these challenges through consideration of alternative outcomes—especially before receiving feedback on actual outcomes—and by calibrating task difficulty with one's knowledge level. As such, we propose that a major function of education—broadly construed as the work of teachers, journalists, parents, etc.—is to assist learners in using their metacognitive surprise signals to facilitate the building and adaptation of belief networks.     

Editors’ Introduction and Review: An Appraisal of Surprise: Tracing the Threads That Stitch It Together

Though the scientific study of surprise dates back to Darwin ( 1872 ), there was an upsurge in interest beginning in the 1960s and 70s, and this has continued to the present. Recent developments have shed much light on the cognitive mechanisms and consequences of surprise, but research has often been siloed within sub‐areas of Cognitive Science. A central challenge for research on surprise is, therefore, to connect various research programs around their overlapping foci. This issue has its roots in a symposium on surprise, entitled “Triangulating Surprise: Expectations, Uncertainty, and Making Sense,” at the 36th Annual Conference of the Cognitive Science Society (Quebec City, July 2014). Building on the interdisciplinary conversations that started at the symposium, this issue aims to draw attention to some promising empirical and modeling results and their theoretical implications. The present paper sets the stage for the issue by presenting a historical summary, discussing contrasting definitions of surprise, and then by tracing major threads that run through both this issue and the larger literature on surprise. Our aim is to develop broader, shared understandings of the main insights, theories, and findings regarding surprise, with a view to supporting future integration and progress.     

The Role of the Anterior Cingulate Cortex in Prediction Error and Signaling Surprise

In the past two decades, reinforcement learning (RL) has become a popular framework for understanding brain function. A key component of RL models, prediction error, has been associated with neural signals throughout the brain, including subcortical nuclei, primary sensory cortices, and prefrontal cortex. Depending on the location in which activity is observed, the functional interpretation of prediction error may change: Prediction errors may reflect a discrepancy in the anticipated and actual value of reward, a signal indicating the salience or novelty of a stimulus, and many other interpretations. Anterior cingulate cortex (ACC) has long been recognized as a region involved in processing behavioral error, and recent computational models of the region have expanded this interpretation to include a more general role for the region in predicting likely events, broadly construed, and signaling deviations between expected and observed events. Ongoing modeling work investigating the interaction between ACC and additional regions involved in cognitive control suggests an even broader role for cingulate in computing a hierarchically structured surprise signal critical for learning models of the environment. The result is a predictive coding model of the frontal lobes, suggesting that predictive coding may be a unifying computational principle across the neocortex.     

Violations of Core Knowledge Shape Early Learning

Research on cognitive development has revealed that even the youngest minds detect and respond to events that adults find surprising. These surprise responses suggest that infants have a basic set of “core” expectations about the world that are shared with adults and other species. However, little work has asked what purpose these surprise responses serve. Here we discuss recent evidence that violations of core knowledge offer special opportunities for learning. Infants and young children make predictions about the world on the basis of their core knowledge of objects, quantities, and social entities. We argue that when these predictions fail to match the observed data, infants and children experience an enhanced drive to seek and retain new information. This impact of surprise on learning is not equipotent. Instead, it is directed to entities that are relevant to the surprise itself; this drive propels children—even infants—to form and test new hypotheses about surprising aspects of the world. We briefly consider similarities and differences between these recent findings with infants and children, on the one hand, and findings on prediction errors in humans and non‐human animals, on the other. These comparisons raise open questions that require continued inquiry, but suggest that considering phenomena across species, ages, kinds of surprise, and types of learning will ultimately help to clarify how surprise shapes thought.     

The Cognitive‐Evolutionary Model of Surprise: A Review of the Evidence

Research on surprise relevant to the cognitive‐evolutionary model of surprise proposed by Meyer, Reisenzein, and Schützwohl (1997) is reviewed. The majority of the assumptions of the model are found empirically supported. Surprise is evoked by unexpected (schema‐discrepant) events and its intensity is determined by the degree if schema‐discrepancy, whereas the novelty and the valence of the eliciting events probably do not have an independent effect. Unexpected events cause an automatic interruption of ongoing mental processes that is followed by an attentional shift and attentional binding to the events, which is often followed by causal and other event analysis processes and by schema revision. The facial expression of surprise postulated by evolutionary emotion psychologists has been found to occur rarely in surprise, for as yet unknown reasons. A physiological orienting response marked by skin conductance increase, heart rate deceleration, and pupil dilation has been observed to occur regularly in the standard version of the repetition‐change paradigm of surprise induction, but the specificity of these reactions as indicators of surprise is controversial. There is indirect evidence for the assumption that the feeling of surprise consists of the direct awareness of the schema‐discrepancy signal, but this feeling, or at least the self‐report of surprise, is also influenced by experienced interference. In contrast, facial feedback probably does contribute substantially to the feeling of surprise and the evidence for the hypothesis that surprise is affected by the difficulty of explaining an unexpected event is, in our view, inconclusive. Regardless of how the surprise feeling is constituted, there is evidence that it has both motivational and informational effects. Finally, the prediction failure implied by unexpected events sometimes causes a negative feeling, but there is no convincing evidence that this is always the case, and we argue that even if it were so, this would not be a sufficient reason for regarding this feeling as a component, rather than as an effect of surprise.     

Seeing Patterns in Randomness: A Computational Model of Surprise

While seemingly a ubiquitous cognitive process, the precise definition and function of surprise remains elusive. Surprise is often conceptualized as being related to improbability or to contrasts with higher probability expectations. In contrast to this probabilistic view, we argue that surprising observations are those that undermine an existing model, implying an alternative causal origin. Surprises are not merely improbable events; instead, they indicate a breakdown in the model being used to quantify probability. We suggest that the heuristic people rely on to detect such anomalous events is randomness deficiency. Specifically, people experience surprise when they identify patterns where their model implies there should only be random noise. Using algorithmic information theory, we present a novel computational theory which formalizes this notion of surprise as randomness deficiency. We also present empirical evidence that people respond to randomness deficiency in their environment and use it to adjust their beliefs about the causal origins of events. The connection between this pattern‐detection view of surprise and the literature on learning and interestingness is discussed.     

Beyond Single-Level Accounts: The Role of Cognitive Architectures in Cognitive Scientific Explanation

We consider approaches to explanation within the cognitive sciences that begin with Marr's computational level (e.g., purely Bayesian accounts of cognitive phenomena) or Marr's implementational level (e.g., reductionist accounts of cognitive phenomena based only on neural-level evidence) and argue that each is subject to fundamental limitations which impair their ability to provide adequate explanations of cognitive phenomena. For this reason, it is argued, explanation cannot proceed at either level without tight coupling to the algorithmic and representation level. Even at this level, however, we argue that additional constraints relating to the decomposition of the cognitive system into a set of interacting subfunctions (i.e., a cognitive architecture) are required. Integrated cognitive architectures that permit abstract specification of the functions of components and that make contact with the neural level provide a powerful bridge for linking the algorithmic and representational level to both the computational level and the implementational level.     

Learning Disabilities in Epilepsy: Neurophysiological Aspects

Summary: Subclinical generalized spike-wave discharges are often accompanied by transitory cognitive impairment, demonstrable by psychological testing during EEG recording. Transitory cognitive impairment is demonstrated most readily by difficult tasks and during generalized regular spike-wave bursts lasting for more than 3 s, but can also be found during briefer and even focal discharges. That this is not simply a consequence of global inattention is shown by the fact that focal discharges exhibit some specificity: left-sided focal spiking is more likely to produce errors on verbal tasks, for instance, whereas right-sided discharges are more often accompanied by impairment in handling nonverbal material. Both learning difficulties in general and specific abnormal patterns of cognitive functioning are well documented in children with epilepsy and are most pronounced in those with frequent interictal discharges. However, there is now evidence that intermittent cognitive impairment due to the discharges themselves contributes significantly to such neurophysiological abnormalities. The significance of transitory cognitive impairment accompanying subclinical EEG discharges for everyday functioning is uncertain, but there is experimental evidence that subclinical discharges may be accompanied by disruption of educational skills in children or by impairment of driving performance in motorists. In some individuals, suppression of discharges by antiepileptic drugs has demonstrably improved psychological function, but further work is required to determine the indications for such treatment     

The Role of the Central Cholinergic Projections in Cognition: Implications of the Effects of Scopolamine on Discrimination Learning by Monkeys

In humans, administration of the cholinergic antagonist scopolamine impairs the encoding of information into long-term memory and has effects on other cognitive processes. It has been supposed that it is inhibition of the rising cholinergic projections from the basal forebrain, specifically from the basal nucleus of Meynert (NBM) to the neocortex and from the medial septum/vertical limb of the diagonal band of Broca (MS/VDB) to the hippocampus, that results in these cognitive impairments. In this paper, we describe the effects of scopolamine treatment in monkeys on learning different sorts of visual discrimination and visuospatial conditional tasks and compare these results to the effects of lesions of the rising cholinergic projections. Experiments in rodents in which these projections have been selectively destroyed have failed to produce a consensus view of the functions of these two areas. In particular, highly specific immunotoxic lesions of the NBM have largely failed to produce changes in task performance that can be interpreted as resulting from a cognitive impairment. In monkeys, lesions of the NBM produce modest or short-lasting, impairments in visual discrimination learning, retention, and reversal, whereas lesions of the MS/VDB produce large and permanent impairments of certain types of conditional learning. Similar impairments produced by scopolamine in monkeys and additive effects of lesions of the NBM or MS/VDB with scopolamine suggest that scopolamine has these effects by acting on the rising cholinergic pathways rather than on other cholinergic systems in the brain. It is argued that the rising cholinergic projections sustain the functions of the target areas; in the case of the hippocampus in humans, the function is usually regarded as being the analysis of information in a way that is pertinent to the formation of episodic memories and in the case of the neocortex, is the analysis of information in a manner that is relevant to the cognitive processing of on-going events and the acquisition of semantic knowledge.     

Negative Affect and Child Internalizing Symptoms: The Mediating Role of Perfectionism

Separate lines of research have linked the temperament factor negative affect and perfectionism with internalizing disorders. Despite theory, no previous studies have connected these lines of research to examine internalizing pathology. The current study tested a path model to investigate the mediating effect of perfectionism domains on the relation between negative affect and child anxiety, worry, and depression symptoms. Participants were 61 parent–child dyads recruited from the community. Children were 7–13 years old (54.1 % male; 88.2 % Caucasian). Overall the model fit the data well. Analyses indicated that separate domains of perfectionism mediated separate relations between negative affect and child anxiety, worry, and depression symptoms. The findings suggest that domains of perfectionism may be implicated in specific paths between negative affect and child anxiety, depression, and worry. Implications and future directions are discussed.     

The Impact of Left Temporal Lobe Seizure Disorder On Learning Disorders: A Case Study

This case study describes the relationship between left temporal lobe epilepsy (TLE) and learning disabilities in a 26-year-old male college student. The client developed seizures following an episode of mycoplasma encephalitis at the age of 7. The client underwent a left temporal lobectomy involving resection of the left mesial temporal lobe, amygdala, hippocampus, and part of the brain stem 6 years prior to the current evaluation, in an attempt to address the frequency of the seizures. The surgery was extensive, including neocortical resection extending posterior to the vein of Labbe along the inferior temporal gyrus. The lobectomy reportedly successfully eliminated the seizures and the need for anti-seizure medications, but no neurological or neuropsychological follow-up occurred until 2009 when he was referred by his academic program for an evaluation of learning disabilities. Results of the neuropsychological evaluation indicated significant expressive language functioning deficits, with generally better-preserved receptive language. However, compared to a pre-surgical neuropsychological evaluation there was evidence for subtle to mild improvement in several aspects of cognitive functioning, likely due to seizure elimination and discontinuation of the anti-seizure medication. Nonetheless, his deficits resulted in significant functional impact on his academic abilities, thus implications for academic intervention were discussed.     

The Myth of Cognitive Decline: Non‐Linear Dynamics of Lifelong Learning

As adults age, their performance on many psychometric tests changes systematically, a finding that is widely taken to reveal that cognitive information‐processing capacities decline across adulthood. Contrary to this, we suggest that older adults' changing performance reflects memory search demands, which escalate as experience grows. A series of simulations show how the performance patterns observed across adulthood emerge naturally in learning models as they acquire knowledge. The simulations correctly identify greater variation in the cognitive performance of older adults, and successfully predict that older adults will show greater sensitivity to fine‐grained differences in the properties of test stimuli than younger adults. Our results indicate that older adults' performance on cognitive tests reflects the predictable consequences of learning on information‐processing, and not cognitive decline. We consider the implications of this for our scientific and cultural understanding of aging.     

Learning and Liking of Melody and Harmony: Further Studies in Artificial Grammar Learning

Much of what we know and love about music is based on implicitly acquired mental representations of musical pitches and the relationships between them. While previous studies have shown that these mental representations of music can be acquired rapidly and can influence preference, it is still unclear which aspects of music influence learning and preference formation. This article reports two experiments that use an artificial musical system to examine two questions: (1) which aspects of music matter most for learning, and (2) which aspects of music matter most for preference formation. Two aspects of music are tested: melody and harmony. In Experiment 1 we tested the learning and liking of a new musical system that is manipulated melodically so that only some of the possible conditional probabilities between successive notes are presented. In Experiment 2 we administered the same tests for learning and liking, but we used a musical system that is manipulated harmonically to eliminate the property of harmonic whole‐integer ratios between pitches. Results show that disrupting melody (Experiment 1) disabled the learning of music without disrupting preference formation, whereas disrupting harmony (Experiment 2) does not affect learning and memory but disrupts preference formation. Results point to a possible dissociation between learning and preference in musical knowledge.     

Psychiatric Comorbidity in Learning Disorder: Analysis of Family Variables

Objective This study aimed to evaluate the role of parental relational styles on the development of psychopathological disturbances in children with Learning Disability (LD). Method Fifty-six children aged 7–12 diagnosed with LD were evaluated on the basis of the Children Behaviour Check List (CBCL) completed by parents. Parents completed an Attachment Style Questionnaire (ASQ) and a family relationships internal representations questionnaire (Family Attitudes Questionnaire, FAQ). Multiple regressions were performed, using parental FAQ and ASQ scores as predictors and children’s CBCL scores as criterion Results Sixty percent of children in the sample obtained a clinical score in at least one of the two syndrome groups. Internalizing behavior in children was predicted by father’s Relationships as Secondary and mother’s Need for Approval and Preoccupation with Relationships scores. Externalizing behavior was predicted by both father and mother’s Need for Approval scores. Conclusions Insecure attachment styles in parents appeared associated with maladaptive emotional–behavioral strategies in their children and may be interpreted as a risk factor for the development of the latter. More specifically, internalizing behavior in children appeared associated with dimensions pointing to dismissing attachment in fathers and to preoccupied attachment in mothers, while externalizing behavior appeared associated with dimensions indicating preoccupied attachment in both parents. Possible interpretations of these data within the LD population are put forward.     

Sequential Expectations: The Role of Prediction‐Based Learning in Language

Prediction‐based processes appear to play an important role in language. Few studies, however, have sought to test the relationship within individuals between prediction learning and natural language processing. This paper builds upon existing statistical learning work using a novel paradigm for studying the on‐line learning of predictive dependencies. Within this paradigm, a new “prediction task” is introduced that provides a sensitive index of individual differences for developing probabilistic sequential expectations. Across three interrelated experiments, the prediction task and results thereof are used to bridge knowledge of the empirical relation between statistical learning and language within the context of nonadjacency processing. We first chart the trajectory for learning nonadjacencies, documenting individual differences in prediction learning. Subsequent simple recurrent network simulations then closely capture human performance patterns in the new paradigm. Finally, individual differences in prediction performances are shown to strongly correlate with participants’ sentence processing of complex, long‐distance dependencies in natural language.     

The Role of Dopamine in Associative Learning in Drosophila: An Updated Unified Model

Learning to associate a positive or negative experience with an unrelated cue after the presentation of a reward or a punishment defines associative learning. The ability to form associative memories has been reported in animal species as complex as humans and as simple as insects and sea slugs. Associative memory has even been reported in tardigrades [1], species that diverged from other animal phyla 500 million years ago. Understanding the mechanisms of memory formation is a fundamental goal of neuroscience research. In this article, we work on resolving the current contradictions between different Drosophila associative memory circuit models and propose an updated version of the circuit model that predicts known memory behaviors that current models do not. Finally, we propose a model for how dopamine may function as a reward prediction error signal in Drosophila, a dopamine function that is well-established in mammals but not in insects [2, 3].     

The Role of Violence Exposure and Negative Affect in Understanding Child and Adolescent Aggression

Aggressive behaviors in youth tend to be relatively stable across the lifespan and are associated with maladaptive functioning later in life. Researchers have recently identified that both violence exposure and negative affective experiences are related to the development of aggressive behaviors. Children exposed to violence also often experience negative affect (NA) in the form of anxiety and depression. Bringing these findings together, the current study used a clinical sample of youth (N = 199; ages 7–17 years) referred to a psychiatric residential treatment facility to examine the specific contributions of NA and exposure to violence on the development of aggressive behaviors in youth. Using structural equation modeling, both NA and recent exposure to violence significantly predicted aggressive behaviors. More importantly, negative affect partially mediated the relationship between exposure to violence and aggression. Implications of these findings from a clinical perspective and future directions for research on aggression are discussed.