An attention-based hybrid deep learning framework integrating brain connectivity and activity of resting-state functional MRI data |
| |
| Affiliation: | 1. Brainnetome Center and National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China;2. School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China;3. Tri-Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State University, Georgia Institute of Technology, Emory University, Atlanta, GA, USA;4. State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, China;5. School of Computer and Information Technology, Shanxi University, Taiyuan, China;1. National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, School of Biomedical Engineering, Health Science Center, Shenzhen University, Shenzhen 518060, China;2. School of Electrical and Electronic Engineering, Nanyang Technological University, 639798, Singapore;3. Guangdong Key Laboratory of Intelligent Information Processing and Shenzhen Key Laboratory of Media Security, College of Information Engineering, Shenzhen University, Shenzhen 518060, China;4. Artificial Intelligence Institute, Hangzhou Dianzi University, Zhejiang 310010, China;1. The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, #2006, Xiyuan Avenue, Chengdu, Sichuan 611731, China;2. Cortical Architecture Imaging and Discovery Lab, Department of Computer Science and Bioimaging Research Center, the University of Georgia, Athens, United States;3. Center for Brain and Brain-Inspired Computing Research, School of Computer Science, Northwestern Polytechnical University, Xi''an, China;4. School of Automation, Northwestern Polytechnical University, Xi''an, China;5. School of Biomedical Engineering & Brain and Mind Centre, The University of Sydney, Camperdown, NSW, Australia;6. Neuroscience Research Institute, Key Laboratory for Neuroscience, Ministry of Education of China; Key Laboratory for Neuroscience, National Committee of Health and Family Planning of China; and Department of neurobiology, School of Basic Medical Sciences, Peking University, Beijing, China;7. Research Unit of NeuroInformation, Chinese Academy of Medical Sciences, 2019RU035, Chengdu 611731, China;8. Department of Computer Science and Engineering, The University of Texas at Arlington, Arlington, TX 76019, United States;1. Department of Biomedical Engineering, National University of Singapore, Singapore;2. NUS (Suzhou) Research Institute, Suzhou, China;3. School of Computer Engineering and Science, Shanghai University, China;4. The N.1 Institute for Health, National University of Singapore, Singapore;5. Institute of Data Science, National University of Singapore, Singapore;6. Department of Biomedical Engineering, The Johns Hopkins University, USA |
| |
| Abstract: | Functional magnetic resonance imaging (fMRI) as a promising tool to investigate psychotic disorders can be decomposed into useful imaging features such as time courses (TCs) of independent components (ICs) and functional network connectivity (FNC) calculated by TC cross-correlation. TCs reflect the temporal dynamics of brain activity and the FNC characterizes temporal coherence across intrinsic brain networks. Both features have been used as input to deep learning approaches with decent results. However, few studies have tried to leverage their complementary information to learn optimal representations at multiple facets. Motivated by this, we proposed a Hybrid Deep Learning Framework integrating brain Connectivity and Activity (HDLFCA) together by combining convolutional recurrent neural network (C-RNN) and deep neural network (DNN), aiming to improve classification accuracy and interpretability simultaneously. Specifically, C-RNNAM was proposed to extract temporal dynamic dependencies with an attention module (AM) to automatically learn discriminative knowledge from TC nodes, while DNN was applied to identify the most group-discriminative FNC patterns with layer-wise relevance propagation (LRP). Then, both prediction outputs were concatenated to build a new feature matrix, generating the final decision by logistic regression. The effectiveness of HDLFCA was validated on both multi-site schizophrenia (SZ, n ∼ 1100) and public autism datasets (ABIDE, n ∼ 1522) by outperforming 12 alternative models at 2.8-8.9% accuracy, including 8 models using either static FNC or TCs and 4 models using dynamic FNC. Appreciable classification accuracy was achieved for HC vs. SZ (85.3%) and HC vs. Autism (72.4%) respectively. More importantly, the most group-discriminative brain regions can be easily attributed and visualized, providing meaningful biological interpretability and highlighting the great potential of the proposed HDLFCA model in the identification of valid neuroimaging biomarkers. |
| |
| Keywords: | |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
