Deep Artificial Neural Networks Reveal a Distributed Cortical Network Encoding Propositional Sentence-Level Meaning

Understanding how and where in the brain sentence-level meaning is constructed from words presents a major scientific challenge. Recent advances have begun to explain brain activation elicited by sentences using vector models of word meaning derived from patterns of word co-occurrence in text corpora. These studies have helped map out semantic representation across a distributed brain network spanning temporal, parietal, and frontal cortex. However, it remains unclear whether activation patterns within regions reflect unified representations of sentence-level meaning, as opposed to superpositions of context-independent component words. This is because models have typically represented sentences as “bags-of-words” that neglect sentence-level structure. To address this issue, we interrogated fMRI activation elicited as 240 sentences were read by 14 participants (9 female, 5 male), using sentences encoded by a recurrent deep artificial neural-network trained on a sentence inference task (InferSent). Recurrent connections and nonlinear filters enable InferSent to transform sequences of word vectors into unified “propositional” sentence representations suitable for evaluating intersentence entailment relations. Using voxelwise encoding modeling, we demonstrate that InferSent predicts elements of fMRI activation that cannot be predicted by bag-of-words models and sentence models using grammatical rules to assemble word vectors. This effect occurs throughout a distributed network, which suggests that propositional sentence-level meaning is represented within and across multiple cortical regions rather than at any single site. In follow-up analyses, we place results in the context of other deep network approaches (ELMo and BERT) and estimate the degree of unpredicted neural signal using an “experiential” semantic model and cross-participant encoding.SIGNIFICANCE STATEMENT A modern-day scientific challenge is to understand how the human brain transforms word sequences into representations of sentence meaning. A recent approach, emerging from advances in functional neuroimaging, big data, and machine learning, is to computationally model meaning, and use models to predict brain activity. Such models have helped map a cortical semantic information-processing network. However, how unified sentence-level information, as opposed to word-level units, is represented throughout this network remains unclear. This is because models have typically represented sentences as unordered “bags-of-words.” Using a deep artificial neural network that recurrently and nonlinearly combines word representations into unified propositional sentence representations, we provide evidence that sentence-level information is encoded throughout a cortical network, rather than in a single region.     

Progression of sleep disturbances in Parkinson’s disease: a 5-year longitudinal study

Journal of Neurology - Sleep disorders can occur in early Parkinson’s disease (PD). However, the relationship between different sleep disturbances and their longitudinal evolution has not...     

Low levels of frailty in HIV-positive older adults on antiretroviral therapy in northern Tanzania

Journal of NeuroVirology - There are over 3 million people in sub-Saharan Africa (SSA) aged 50 and over living with HIV. HIV and combined antiretroviral therapy (cART) exposure may accelerate the...     

Three-dimensional visualization of the bovine cardiac conduction system and surrounding structures compared to the arrangements in the human heart

In the human heart, the atrioventricular node is located toward the apex of the triangle of Koch, which is also at the apex of the inferior pyramidal space. It is adjacent to the atrioventricular portion of the membranous septum, through which it penetrates to become the atrioventricular bundle. Subsequent to its penetration, the conduction axis is located on the crest of the ventricular septum, sandwiched between the muscular septum and ventricular component of the membranous septum, where it gives rise to the ramifications of the left bundle branch. In contrast, the bovine conduction axis has a long non-branching component, which penetrates into a thick muscular atrioventricular septum having skirted the main cardiac bone and the rightward half of the non-coronary sinus of the aortic root. It commonly gives rise to both right and left bundle branches within the muscular ventricular septum. Unlike the situation in man, the left bundle branch is long and thin before it branches into its fascicles. These differences from the human heart, however, have yet to be shown in three-dimensions relative to the surrounding structures. We have now achieved this goal by injecting contrast material into the insulating sheaths that surround the conduction network, evaluating the results by subsequent computed tomography. The fibrous atrioventricular membranous septum of the human heart is replaced in the ox by the main cardiac bone and the muscular atrioventricular septum. The apex of the inferior pyramidal space, which in the bovine, as in the human, is related to the atrioventricular node, is placed inferiorly relative to the left ventricular outflow tract. The bovine atrioventricular conduction axis, therefore, originates from a node itself located inferiorly compared to the human arrangement. The axis must then skirt the non-coronary sinus of the aortic root prior to penetrating the thicker muscular ventricular septum, thus accounting for its long non-branching course. We envisage that our findings will further enhance comparative anatomical research.     

Developing Signs of Safety: A Deaf-accessible counselling toolkit for trauma and addiction

The U.S. Deaf community—more than half a million Americans who communicate using American Sign Language (ASL)—experiences higher rates of trauma exposure and substance use disorder (SUD) than the general population. Yet there are no evidence-based treatments for any behavioural health condition that have been evaluated for use with Deaf people. The driving aim of our work, therefore, has been to develop and formally evaluate a Deaf-accessible trauma/SUD counselling approach. Here we describe our initial intervention development work and a single-arm pilot that evaluated the feasibility, acceptability, and preliminary clinical efficacy of Signs of Safety—a Deaf-accessible toolkit to be used with an existing, widely adopted protocol for trauma and addiction (Seeking Safety). Preliminary efficacy results indicated clinically significant reductions in PTSD symptoms and frequency of alcohol use for the Seeking Safety/Signs of Safety model. Frequency of drug use did not change significantly—likely attributable to the mid-study legalization of recreational marijuana in our state. Next steps include the redesign and refilming of Signs of Safety based on pilot participant feedback, again using a Deaf-engaged development and production process. This new toolkit will be tested via a pilot randomized controlled trial designed based on present methodological lessons learned.     

HLA class I loss in colorectal cancer: implications for immune escape and immunotherapy

T cell-mediated immune therapies have emerged as a promising treatment modality in different malignancies including colorectal cancer (CRC). However, only a fraction of patients currently respond to treatment. Understanding the lack of responses and finding biomarkers with predictive value is of great importance. There is evidence that CRC is a heterogeneous disease and several classification systems have been proposed that are based on genomic instability, immune cell infiltration, stromal content and molecular subtypes of gene expression. Human leukocyte antigen class I (HLA-I) plays a pivotal role in presenting processed antigens to T lymphocytes, including tumour antigens. These molecules are frequently lost in different types of cancers, including CRC, resulting in tumour immune escape from cytotoxic T lymphocytes during the natural history of cancer development. The aim of this review is to (i) summarize the prevalence and molecular mechanisms behind HLA-I loss in CRC, (ii) discuss HLA-I expression/loss in the context of the newly identified CRC molecular subtypes, (iii) analyze the HLA-I phenotypes of CRC metastases disseminated via blood or the lymphatic system, (iv) discuss strategies to recover/circumvent HLA-I expression/loss and finally (v) review the role of HLA class II (HLA-II) in CRC prognosis.     

Applying synthetic radiography to intraoral tomosynthesis: a step towards achieving 3D imaging in the dental clinic

Objectives:A practical approach to three-dimensional (3D) intraoral imaging would have many potential applications in clinical dentistry. Stationary intraoral tomosynthesis (sIOT) is an experimental 3D imaging technology that holds promise. The purpose of this study was to explore synthetic radiography as a tool to improve the clinical utility of the images generated by an sIOT scan.Methods:Extracted tooth specimens containing either caries adjacent to restorations (CAR) or vertical root fractures (VRF) were imaged by sIOT and standard dental radiography devices. Qualitative assessments were used to compare the conspicuity of these pathologies in the standard radiographs and in a set of multi-view synthetic radiographs generated from the information collected by sIOT.Results:The sIOT-based synthetic 2D radiographs contained less artefact than the image slices in the reconstructed 3D stack, which is the conventional approach to displaying information from a tomosynthesis scan. As a single sIOT scan can be used to generate synthetic radiographs from multiple viewing angles, the interproximal space was less likely to be obscured in the synthetic images compared to the standard radiograph. Additionally, the multi-view synthetic radiographs can potentially improve the display of CAR and VRFs as compared to a single standard radiograph.Conclusions:This preliminary experience combining synthetic radiography and sIOT in extracted tooth models is encouraging and supports the ongoing study of this promising approach to 3D intraoral imaging with many potential applications.