Adults with Autism Show Increased Sensitivity to Outcomes at Low Error Rates During Decision-Making

Decision-making is an important function that can be quantified using a two-choice prediction task. Individuals with Autistic Disorder (AD) often show highly restricted and repetitive behavior that may interfere with adaptive decision-making. We assessed whether AD adults showed repetitive behavior on the choice task that was unaffected by changing task demands, by examining the influence of experimenter-determined error rates on decision-making. Sixteen AD adults and 14 typically developed subjects were administered a two-choice task using three error rate conditions. Although AD subjects showed occurrences of stereotyped responding, their decision-making behavior was strongly affected by changes in task demands, especially when they experienced frequent success. Thus, behavioral paradigms that provide frequent reinforcement may be helpful in modifying decision-making abilities in AD.     

Impaired spatial and non-spatial configural learning in patients with hippocampal pathology

The hippocampus has been proposed to play a critical role in memory through its unique ability to bind together the disparate elements of an experience. This hypothesis has been widely examined in rodents using a class of tasks known as "configural" or "non-linear", where outcomes are determined by specific combinations of elements, rather than any single element alone. On the basis of equivocal evidence that hippocampal lesions impair performance on non-spatial configural tasks, it has been proposed that the hippocampus may only be critical for spatial configural learning. Surprisingly few studies in humans have examined the role of the hippocampus in solving configural problems. In particular, no previous study has directly assessed the human hippocampal contribution to non-spatial and spatial configural learning, the focus of the current study. Our results show that patients with primary damage to the hippocampus bilaterally were similarly impaired at configural learning within both spatial and non-spatial domains. Our data also provide evidence that residual configural learning can occur in the presence of significant hippocampal dysfunction. Moreover, evidence obtained from a post-experimental debriefing session suggested that patients acquired declarative knowledge of the underlying task contingencies that corresponded to the best-fit strategy identified by our strategy analysis. In summary, our findings support the notion that the hippocampus plays an important role in both spatial and non-spatial configural learning, and provide insights into the role of the medial temporal lobe (MTL) more generally in incremental reinforcement-driven learning.     

Long-lasting decreases in cocaine-reinforced behavior following treatment with the cholinesterase inhibitor tacrine in rats selectively bred for drug self-administration

Tacrine is a centrally acting, reversible cholinesterase inhibitor that increases synaptic levels of acetylcholine (ACh) and can potentiate the actions of dopamine (DA). The present study was conducted to evaluate effects of tacrine on cocaine-reinforced responding in a rat line selectively bred for high levels of drug self-administration (the HS line). HS rats self-administered different doses of cocaine under a fixed-ratio-5 (FR-5) schedule. Over a four-day period, vehicle or tacrine (1.0, 3.2, or 10 mg/kg-day) was infused when animals were maintained in home cages (21 h per day). Tacrine dose-dependently decreased cocaine self-administration. Actions of tacrine differed for self-administration that was initiated within 20 min of pretreatment (described as early sessions), and for self-administration that occurred between one and three days after administration of tacrine was discontinued (late sessions). Tacrine's potency for attenuating self-administration during late sessions was greater for cocaine- relative to food-reinforcement in HS rats, and for HS relative to outbred rats. In a subset of tacrine-treated HS rats, cocaine self-administration was persistently attenuated by more than 80% from pretreatment baseline levels over a one-week period during which no further tacrine was administered. In summary, pretreatment with tacrine can produce a long-lasting attenuation of cocaine-reinforced responding.     

Goal-directed learning of features and forward models

The brain is able to perform actions based on an adequate internal representation of the world, where task-irrelevant features are ignored and incomplete sensory data are estimated. Traditionally, it is assumed that such abstract state representations are obtained purely from the statistics of sensory input for example by unsupervised learning methods. However, more recent findings suggest an influence of the dopaminergic system, which can be modeled by a reinforcement learning approach. Standard reinforcement learning algorithms act on a single layer network connecting the state space to the action space. Here, we involve in a feature detection stage and a memory layer, which together, construct the state space for a learning agent. The memory layer consists of the state activation at the previous time step as well as the previously chosen action. We present a temporal difference based learning rule for training the weights from these additional inputs to the state layer. As a result, the performance of the network is maintained both, in the presence of task-irrelevant features, and at randomly occurring time steps during which the input is invisible. Interestingly, a goal-directed forward model emerges from the memory weights, which only covers the state–action pairs that are relevant to the task. The model presents a link between reinforcement learning, feature detection and forward models and may help to explain how reward systems recruit cortical circuits for goal-directed feature detection and prediction.     

阳性强化疗法在慢性精神分裂症患者工娱治疗中的运用

目的探讨阳性强化疗法与工娱治疗结合对慢性精神分裂症的治疗效果。方法60例慢性精神分裂症患者被随机分为2组,分别给予工娱治疗＋阳性强化疗法和工娱治疗3个月,采用阴性症状评定量表（SANS）,慢性住院精神病评定量表（Mallchester）和现场观察评分,分别于治疗前后评定。结果3个月后治疗组除在情意平淡,意志缺乏,阳性症状因子分外,其于各项评分均显著下降（P〈0．01）,对照组各评分下降不量著（P〉0．05）。结论在工娱治疗时给予阳性强化疗法能明王提高治疗效果,改善病人的社会功能。     

The Cerebellum Is Involved in Reward-based Reversal Learning

The cerebellum has recently been discussed in terms of a possible involvement in reward-based associative learning. To clarify the cerebellar contribution, eight patients with focal vascular lesions of the cerebellum and a group of 24 healthy subjects matched for age and IQ were compared on a range of different probabilistic outcome-based associative learning tasks, assessing acquisition, reversal, cognitive transfer, and generalization as well as the effect of reward magnitude. Cerebellar patients showed intact acquisition of stimulus contingencies, while reward-based reversal learning was significantly impaired. In addition, the patients showed slower acquisition of new stimulus contingencies in a second reward-based learning task, which might reflect reduced carry-over effects. Reward magnitude affected learning only during initial acquisition, with better learning on trials with high rewards in patients and control subjects. Overall, the findings suggest that the cerebellum is implicated in reversal learning as a dissociable component of reward-based learning.     

Differential responsiveness to caffeine and perceived effects of caffeine in moderate and high regular caffeine consumers

Rationale Individual differences in responsiveness to caffeine occur even within a caffeine-consuming population, but the factors that mediate differential responsiveness remain unclear. Objectives To compare caffeine’s effects on performance and mood in a group of high vs moderate consumers of caffeine and to examine the potential role of subjective awareness of the effects of caffeine in mediating any differential responsiveness. Materials and methods Two groups of regular caffeine consumers (<200 mg/day and >200 mg/day) attended two sessions at which mood and cognitive functions were measured before and 30 min after consumption of 400-mg caffeine or placebo in a capsule. Cognitive tests included visual information processing, match-to-sample visual search (MTS) and simple and choice reaction times. Post-session questionnaires asked participants to describe any perceived effect of capsule consumption. Results High consumers, but not moderate consumers, demonstrated significantly faster simple and choice reaction times after caffeine relative to placebo. These effects were not attributable to obvious group differences in withdrawal or tolerance because there were no group differences in baseline mood or in reports of negative affect after caffeine. Instead, the high consumers were more likely to report experiencing positive effects of caffeine, whereas the moderate consumers were more likely to report no effect. Conclusions The sensitivity of caffeine consumers to the mood- and performance-enhancing effects of caffeine is related to their levels of habitual intake. High caffeine consumers are more likely than moderate consumers to perceive broadly positive effects of caffeine, and this may contribute to their levels of use.     

Continuous exposure to the competitive <Emphasis Type="BoldItalic">N</Emphasis>-methyl-<Emphasis Type="SmallCaps">d</Emphasis>-aspartate receptor antagonist,LY235959, facilitates escalation of cocaine consumption in Sprague–Dawley rats

Rationale Chronic high dose consumption of cocaine is associated with significant negative effects to individual users and society. Nevertheless, the precise mechanisms that mediate increases in cocaine consumption in a drug-using individual are not fully understood. Objectives This study used a long access version of the drug self-administration procedure to determine whether escalation of cocaine consumption is mediated by increased activity through N-methyl-d-aspartate (NMDA) receptors. Materials and methods Male Sprague–Dawley rats (n = 63) were first trained to self-administer cocaine (0.33 mg/infusion, i.v.) under a fixed-ratio 1 schedule of reinforcement. After training, some rats were implanted with subcutaneous osmotic minipumps filled with vehicle or the competitive NMDA receptor antagonist, LY235959, and subsequently allowed to self-administer cocaine in short (2 h) or long (6 h) access self-administration sessions. Results Vehicle-treated rats escalated cocaine self-administration across 14 long-access self-administration sessions. Rats treated with LY235959 via osmotic minipump, but not twice daily injections, escalated cocaine self-administration at a greater rate and to a greater degree than vehicle-treated rats. In post-escalation cocaine dose-infusion tests, rats treated continuously with LY235959 self-administered more cocaine (0.08–1.32 mg/infusion) than vehicle-treated rats, regardless of access condition, shifting the dose-infusion curves upward. During extinction sessions, which were conducted after the escalation phase of the study, rats that had long (6 h) access to cocaine stopped responding sooner than rats that had short (2 h) access to cocaine, independent of LY235959 treatment. Conclusions These data are consistent with hypo-glutamatergic consequences of repeated cocaine exposure. Preliminary reports of these data were presented at the 2003 meeting of the Society for Neuroscience (New Orleans, LA) and the 2005 meeting of the College on Problems of Drug Dependence (Orlando, FL).     

A reinforcement learning approach for VQA validation: An application to diabetic macular edema grading

Recent advances in machine learning models have greatly increased the performance of automated methods in medical image analysis. However, the internal functioning of such models is largely hidden, which hinders their integration in clinical practice. Explainability and trust are viewed as important aspects of modern methods, for the latter’s widespread use in clinical communities. As such, validation of machine learning models represents an important aspect and yet, most methods are only validated in a limited way. In this work, we focus on providing a richer and more appropriate validation approach for highly powerful Visual Question Answering (VQA) algorithms. To better understand the performance of these methods, which answer arbitrary questions related to images, this work focuses on an automatic visual Turing test (VTT). That is, we propose an automatic adaptive questioning method, that aims to expose the reasoning behavior of a VQA algorithm. Specifically, we introduce a reinforcement learning (RL) agent that observes the history of previously asked questions, and uses it to select the next question to pose. We demonstrate our approach in the context of evaluating algorithms that automatically answer questions related to diabetic macular edema (DME) grading. The experiments show that such an agent has similar behavior to a clinician, whereby asking questions that are relevant to key clinical concepts.     

Diminished reinforcement sensitivity in adolescence is associated with enhanced response switching and reduced coding of choice probability in the medial frontal pole

Precisely charting the maturation of core neurocognitive functions such as reinforcement learning (RL) and flexible adaptation to changing action-outcome contingencies is key for developmental neuroscience and adjacent fields like developmental psychiatry. However, research in this area is both sparse and conflicted, especially regarding potentially asymmetric development of learning for different motives (obtain wins vs avoid losses) and learning from valenced feedback (positive vs negative). In the current study, we investigated the development of RL from adolescence to adulthood, using a probabilistic reversal learning task modified to experimentally separate motivational context and feedback valence, in a sample of 95 healthy participants between 12 and 45. We show that adolescence is characterized by enhanced novelty seeking and response shifting especially after negative feedback, which leads to poorer returns when reward contingencies are stable. Computationally, this is accounted for by reduced impact of positive feedback on behavior. We also show, using fMRI, that activity of the medial frontopolar cortex reflecting choice probability is attenuated in adolescence. We argue that this can be interpreted as reflecting diminished confidence in upcoming choices. Interestingly, we find no age-related differences between learning in win and loss contexts.