Deep Artificial Neural Networks Reveal a Distributed Cortical Network Encoding Propositional Sentence-Level Meaning

Understanding how and where in the brain sentence-level meaning is constructed from words presents a major scientific challenge. Recent advances have begun to explain brain activation elicited by sentences using vector models of word meaning derived from patterns of word co-occurrence in text corpora. These studies have helped map out semantic representation across a distributed brain network spanning temporal, parietal, and frontal cortex. However, it remains unclear whether activation patterns within regions reflect unified representations of sentence-level meaning, as opposed to superpositions of context-independent component words. This is because models have typically represented sentences as “bags-of-words” that neglect sentence-level structure. To address this issue, we interrogated fMRI activation elicited as 240 sentences were read by 14 participants (9 female, 5 male), using sentences encoded by a recurrent deep artificial neural-network trained on a sentence inference task (InferSent). Recurrent connections and nonlinear filters enable InferSent to transform sequences of word vectors into unified “propositional” sentence representations suitable for evaluating intersentence entailment relations. Using voxelwise encoding modeling, we demonstrate that InferSent predicts elements of fMRI activation that cannot be predicted by bag-of-words models and sentence models using grammatical rules to assemble word vectors. This effect occurs throughout a distributed network, which suggests that propositional sentence-level meaning is represented within and across multiple cortical regions rather than at any single site. In follow-up analyses, we place results in the context of other deep network approaches (ELMo and BERT) and estimate the degree of unpredicted neural signal using an “experiential” semantic model and cross-participant encoding.SIGNIFICANCE STATEMENT A modern-day scientific challenge is to understand how the human brain transforms word sequences into representations of sentence meaning. A recent approach, emerging from advances in functional neuroimaging, big data, and machine learning, is to computationally model meaning, and use models to predict brain activity. Such models have helped map a cortical semantic information-processing network. However, how unified sentence-level information, as opposed to word-level units, is represented throughout this network remains unclear. This is because models have typically represented sentences as unordered “bags-of-words.” Using a deep artificial neural network that recurrently and nonlinearly combines word representations into unified propositional sentence representations, we provide evidence that sentence-level information is encoded throughout a cortical network, rather than in a single region.     

Stroke care in Italy at the time of the COVID-19 pandemic: a lesson to learn

Journal of Neurology - From March to May 2020, the Italian health care system, as many others, was almost entirely devoted to the fight against the COVID-19 pandemic. In this context, a number of...     

Cognitive and behavioral manifestations in SARS-CoV-2 infection: not specific or distinctive features?

Neurological Sciences - Patients with COVID-19 are increasingly reported to suffer from a wide range of neurological complications, affecting both the central and peripheral nervous system. Among...     

Description of fecal shedding of Cryptosporidium parvum oocysts in experimentally challenged dairy calves

The objective was to describe the probability of Cryptosporidium parvum fecal oocyst shedding at different magnitudes of exposure, the pattern of fecal shedding over time, and factors affecting fecal shedding in dairy calves. Within the first 24 h of life, 36 calves were experimentally challenged with C. parvum oocysts at one of four possible magnitudes of oral exposure (1?×?10³, 1?×?10⁴, 1?×?10⁵, and 1?×?10⁶ oocysts), and 7 control calves were sham dosed. Fecal shedding occurred in 33 (91.7 %) experimentally challenged calves and in none of the control calves. There was a difference in the log-total number of oocysts counted per gram of feces dry weight among the four exposure groups; calves with the lowest magnitude of exposure (1?×?10³ oocysts) shed less than the other three groups. At higher magnitudes of exposure, there was more variability in the range of fecal oocyst shedding. There was an inverse relationship between the log-total amount of oocysts counted per gram of feces dry weight and the number of days to the onset of fecal shedding per calf, i.e., the more time that elapsed to the onset of fecal shedding, the fewer oocysts that were shed. The pattern of fecal shedding over time for all calves shedding oocysts was curvilinear; the number of oocysts increased with time, reached a peak, and declined. Therefore, the dynamics of oocyst shedding can be influenced in part by limiting exposure among calves and delaying the onset of fecal oocyst shedding.     

Pro-caspase-3 is constitutively expressed in luteinized granulosa cells from women undergoing controlled ovarian stimulation for in vitro fertilization

Apoptosis regulation in luteinized granulosa cells (LGC) during assisted reproduction procedures is still controversial. Caspase-3 is a major apoptosis mediator encoded by CASP3 and formed through cleavage of its precursor pro-caspase-3. The aim of this study was to investigate the expression patterns of pro-caspase-3 (mRNA and protein) and cleaved caspase-3 in human LGC. Thirty-five women undergoing controlled ovarian stimulation for in vitro fertilization were prospectively enrolled in the study. LGC were isolated from follicular fluid during oocyte pickup and evaluated by immunocytochemistry for pro-caspase-3 and cleaved caspase-3, and by real-time PCR for CASP3 mRNA expression. We found a positive staining of pro-caspase-3 in 77 % of the LGC (95 % confidence interval [CI] 60%–84%), whereas cleaved caspase-3 was found in only 4% of the cells (95 % CI 3%–6%). The abundance of cells expressing pro-caspase-3 was independent from CASP3 mRNA levels (r = 0.24, p = 0.255) and did not correlate with the amount of cleaved caspase-3 (r = -0.24, p = 0.186). Multivariable logistic regression showed that pro-caspase-3 positivity was not influenced by clinical characteristics such as age, cause or length of infertility, antral follicle count or hormonal drugs used to induce ovulation. These findings suggest that pro-caspase-3 is constitutively expressed in LGC, allowing quick cleavage into active caspase-3 and apoptosis triggering whenever needed in the course of gonadotropin-induced follicular development.     

Tooth loss in complying and non-complying periodontitis patients with different periodontal risk levels during supportive periodontal care

Clinical Oral Investigations - To evaluate yearly tooth loss rate (TLR) in periodontitis patients with different periodontal risk levels who had complied or not complied with supportive periodontal...     

Single flap approach with or without enamel matrix derivative in the treatment of severe supraosseous defects: a retrospective study

Clinical Oral Investigations - To comparatively evaluate the clinical effectiveness of the single flap approach (SFA) with and without enamel matrix derivative (EMD) in the treatment of...     

Genetic variation involved in the risk to external apical root resorption in orthodontic patients: a systematic review

Clinical Oral Investigations - To perform a systematic review/meta-analysis to elucidate the scientific basis for the association between genetic variations and risk of external apical root...