Benchmarking weakly-supervised deep learning pipelines for whole slide classification in computational pathology
Affiliation:
1. Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany;2. Department of Pathology, Brigham and Women''s Hospital, Harvard Medical School, Boston, MA, USA.;3. Institute of Pathology, University of Bern, Switzerland.;4. Institute of Pathology and Molecular Pathology, Kepler University Hospital, Johannes Kepler University Linz, Linz, Austria;5. Institute of Pathology, University Hospital RWTH Aachen, Aachen, Germany;6. Department of Pathology, GROW School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht, The Netherlands.;7. Division of Pathology and Data Analytics, Leeds Institute of Medical Research at St James''s, University of Leeds, Leeds, UK;8. Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany;9. Division of Preventive Oncology, German Cancer Research Center (DKFZ) and National Center for Tumor Diseases (NCT), Heidelberg, Germany;10. German Cancer Consortium (DKTK), German Cancer Research Center (DKFZ), Heidelberg, Germany;11. Division of Cancer Epidemiology, German Cancer Research Center (DKFZ), Heidelberg, Germany;12. Cancer Epidemiology Group, University Cancer Center Hamburg, University Medical Center Hamburg-Eppendorf, Hamburg, Germany;13. Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany;14. Department of Radiology, University Hospital RWTH Aachen, Aachen, Germany;15. Department of Physics of Molecular Imaging Systems, Experimental Molecular Imaging, RWTH Aachen University, Aachen, Germany;p. Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany;q. Comprehensive Diagnostic Center Aachen (CDCA), University Hospital Aachen, Aachen, Germany;r. Hyperion Hybrid Imaging Systems GmbH, Aachen, Germany;s. Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany;1. Physical Sciences, Sunnybrook Research Institute, Toronto, Canada;2. Department of Medical Biophysics, University of Toronto, Canada;3. Department of Computer Science, University of Toronto, Canada;4. Department of Electrical & Computer Engineering, University of Toronto, Canada;1. IBM Zurich Research Lab, Zurich, Switzerland;2. Computer-Assisted Applications in Medicine, ETH Zurich, Zurich, Switzerland;3. Signal Processing Laboratory 5, EPFL, Lausanne, Switzerland;4. National Cancer Institute - IRCCS-Fondazione Pascale, Naples, Italy;5. Institute for High Performance Computing and Networking - CNR, Naples, Italy;6. Aurigen- Centre de Pathologie, Lausanne, Switzerland;7. Lausanne University Hospital, Lausanne, Switzerland;8. Department of Information Technology, Uppsala University, Sweden;1. Kimia Lab, University of Waterloo, Waterloo, ON, Canada;2. Vector Institute, MaRS Centre, Toronto, ON, Canada;3. McMaster University, Hamilton, ON, Canada;4. Artificial Intelligence and Informatics, Mayo Clinic, Rochester, MN, USA;1. Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing 100191, China;2. Image Processing Center, School of Astronautics, Beihang University, Beijing 102206, China;3. Department of Pathology, Tianjin Fifth Central Hospital, Tianjin 300450, China;4. School of Software, Hefei University of Technology, Hefei 230601, China;5. Wankangyuan Tianjin Gene Technology, Inc, Tianjin 300220, China;6. Tianjin Institute of Industrial Biotechnology, Chinese Academy of Sciences, Tianjin 300308, China
Abstract:
Artificial intelligence (AI) can extract visual information from histopathological slides and yield biological insight and clinical biomarkers. Whole slide images are cut into thousands of tiles and classification problems are often weakly-supervised: the ground truth is only known for the slide, not for every single tile. In classical weakly-supervised analysis pipelines, all tiles inherit the slide label while in multiple-instance learning (MIL), only bags of tiles inherit the label. However, it is still unclear how these widely used but markedly different approaches perform relative to each other.We implemented and systematically compared six methods in six clinically relevant end-to-end prediction tasks using data from N=2980 patients for training with rigorous external validation. We tested three classical weakly-supervised approaches with convolutional neural networks and vision transformers (ViT) and three MIL-based approaches with and without an additional attention module. Our results empirically demonstrate that histological tumor subtyping of renal cell carcinoma is an easy task in which all approaches achieve an area under the receiver operating curve (AUROC) of above 0.9. In contrast, we report significant performance differences for clinically relevant tasks of mutation prediction in colorectal, gastric, and bladder cancer. In these mutation prediction tasks, classical weakly-supervised workflows outperformed MIL-based weakly-supervised methods for mutation prediction, which is surprising given their simplicity. This shows that new end-to-end image analysis pipelines in computational pathology should be compared to classical weakly-supervised methods. Also, these findings motivate the development of new methods which combine the elegant assumptions of MIL with the empirically observed higher performance of classical weakly-supervised approaches. We make all source codes publicly available at https://github.com/KatherLab/HIA, allowing easy application of all methods to any similar task.
