CS-CO: A Hybrid Self-Supervised Visual Representation Learning Method for H&E-stained Histopathological Images
Institution:
1. Ministry of Education Key Laboratory of Bioinformatics, Bioinformatics Division, Beijing National Research Center for Information Science and Technology, Department of Automation, Tsinghua University, Beijing 100084, China;2. Department of Pathology, Xiangya Hospital, Central South University, 87 Xiangya Road, Changsha, 410078, Hunan, China;3. Department of Pathology, School of Basic Medical Science, Central South University, 172 Tongzi Road, Changsha, 410013, Hunan, China;1. School of Electrical & Computer Engineering, Cornell University, USA;2. Institute for Infocomm Research (I2R), A*STAR, Singapore;3. Department of Radiology, Weill Cornell Medicine, USA;1. Department of Bioengineering, University of Louisville, Louisville, KY, USA;2. Department of Pathology and Laboratory Medicine, University of Louisville, Louisville, KY, USA;1. Department of Electrical Engineering, ESAT/PSI, KU Leuven, Leuven, Belgium;2. Department of Imaging and Pathology, Radiology, KU Leuven, Leuven, Belgium;3. Medical Imaging Research Center, UZ Leuven, Herestraat 49 - 7003, Leuven 3000, Belgium;1. Physical Sciences, Sunnybrook Research Institute, Toronto, Canada;2. Department of Medical Biophysics, University of Toronto, Canada;3. Department of Computer Science, University of Toronto, Canada;4. Department of Electrical & Computer Engineering, University of Toronto, Canada;1. Intelligent Modeling and Analysis Group, School of Computer Science, University of Nottingham, UK;2. The Sir Peter Mansfield Imaging Centre, University of Nottingham, UK
Abstract:
Visual representation extraction is a fundamental problem in the field of computational histopathology. Considering the powerful representation capacity of deep learning and the scarcity of annotations, self-supervised learning has emerged as a promising approach to extract effective visual representations from unlabeled histopathological images. Although a few self-supervised learning methods have been specifically proposed for histopathological images, most of them suffer from certain defects that may hurt the versatility or representation capacity. In this work, we propose CS-CO, a hybrid self-supervised visual representation learning method tailored for H&E-stained histopathological images, which integrates advantages of both generative and discriminative approaches. The proposed method consists of two self-supervised learning stages: cross-stain prediction (CS) and contrastive learning (CO). In addition, a novel data augmentation approach named stain vector perturbation is specifically proposed to facilitate contrastive learning. Our CS-CO makes good use of domain-specific knowledge and requires no side information, which means good rationality and versatility. We evaluate and analyze the proposed CS-CO on three H&E-stained histopathological image datasets with downstream tasks of patch-level tissue classification and slide-level cancer prognosis and subtyping. Experimental results demonstrate the effectiveness and robustness of the proposed CS-CO on common computational histopathology tasks. Furthermore, we also conduct ablation studies and prove that cross-staining prediction and contrastive learning in our CS-CO can complement and enhance each other. Our code is made available at https://github.com/easonyang1996/CS-CO.
