Computational tissue staining of non-linear multimodal imaging using supervised and unsupervised deep learning

Hematoxylin and Eosin (H&E) staining is the ’gold-standard’ method in histopathology. However, standard H&E staining of high-quality tissue sections requires long sample preparation times including sample embedding, which restricts its application for ’real-time’ disease diagnosis. Due to this reason, a label-free alternative technique like non-linear multimodal (NLM) imaging, which is the combination of three non-linear optical modalities including coherent anti-Stokes Raman scattering, two-photon excitation fluorescence and second-harmonic generation, is proposed in this work. To correlate the information of the NLM images with H&E images, this work proposes computational staining of NLM images using deep learning models in a supervised and an unsupervised approach. In the supervised and the unsupervised approach, conditional generative adversarial networks (CGANs) and cycle conditional generative adversarial networks (cycle CGANs) are used, respectively. Both CGAN and cycle CGAN models generate pseudo H&E images, which are quantitatively analyzed based on mean squared error, structure similarity index and color shading similarity index. The mean of the three metrics calculated for the computationally generated H&E images indicate significant performance. Thus, utilizing CGAN and cycle CGAN models for computational staining is beneficial for diagnostic applications without performing a laboratory-based staining procedure. To the author’s best knowledge, it is the first time that NLM images are computationally stained to H&E images using GANs in an unsupervised manner.     

Influence of hematoxylin and eosin staining on the quantitative analysis of second harmonic generation imaging of fixed tissue sections

Second harmonic generation (SHG) microscopy has emerged over the past two decades as a powerful tool for tissue characterization and diagnostics. Its main applications in medicine are related to mapping the collagen architecture of in-vivo, ex-vivo and fixed tissues based on endogenous contrast. In this work we present how H&E staining of excised and fixed tissues influences the extraction and use of image parameters specific to polarization-resolved SHG (PSHG) microscopy, which are known to provide quantitative information on the collagen structure and organization. We employ a theoretical collagen model for fitting the experimental PSHG datasets to obtain the second order susceptibility tensor elements ratios and the fitting efficiency. Furthermore, the second harmonic intensity acquired under circular polarization is investigated. The evolution of these parameters in both forward- and backward-collected SHG are computed for both H&E-stained and unstained tissue sections. Consistent modifications are observed between the two cases in terms of the fitting efficiency and the second harmonic intensity. This suggests that similar quantitative analysis workflows applied to PSHG images collected on stained and unstained tissues could yield different results, and hence affect the diagnostic accuracy.     

Diagnosis of BK Viral Nephropathy in the Renal Allograft Biopsy: Role of Fluorescence in Situ Hybridization

Early recognition of BK viral nephropathy is essential for successful management. Our aim in this study was to evaluate a novel fluorescence in situ hybridization (FISH) assay for detection of BK virus in renal transplant biopsies in the context of standard detection methods. Renal allograft biopsies (n = 108) were analyzed via H&E, immunohistochemistry (IHC) for simian virus 40, and FISH for BK virus. BK virus was detected in 16 (14.8%) cases by H&E, 13 (12%) cases by IHC, 18 (16.6%) cases by FISH, and 19 (17.6%) cases by real-time PCR; 24 of 108 showed a discrepancy in ≥1 testing modalities. Comparison of H&E, IHC, and FISH showed no statistical difference in detection of BK virus. However, performing comparisons between the different tissue-based assays in the context of plasma or urine real-time PCR results showed significant improvement in detection of BK by FISH over H&E (P = 0.02) but not IHC (P = 0.07). This novel FISH-based approach for BK virus identification in renal allograft biopsy tissue mirrored real-time PCR results and showed superior performance to detection of inclusions by H&E. Therefore, use of FISH for BK virus detection in the setting of renal allograft biopsy is a useful and sensitive detection method and could be adopted in any laboratory that currently performs FISH analysis.     

A novel method of virtual histopathology using laser-scanning confocal microscopy in-vitro with untreated fresh specimens from the gastrointestinal mucosa

BACKGROUND AND STUDY AIMS: Histopathological examination for superficial gastrointestinal lesions has been mainly based upon the light microscopic examination of thin-slice specimens with hematoxylin and eosin (H&E) staining. However, it takes at least a couple of days to create a slide-glass for microscopic study. In order to obtain immediate microscopic images for untreated specimens, the authors used laser-scanning confocal microscopy (LCM) to study fresh samples of gastrointestinal mucosa. MATERIALS AND METHODS: Fresh untreated mucosal specimens from the esophagus, stomach, and colon, obtained by endoscopic pinch biopsy, polypectomy, or endoscopic mucosal resection (EMR), were fixed in normal saline and examined by LCM collecting the reflective light of a 488-nm wavelength argon laser beam. Findings from the LCM image were compared with those of conventional H&E staining in all specimens. For objective evaluation of the similarity of both pictures, the nucleus-to-cytoplasm ratio (N/C) of normal mucosa and that of cancer of the esophagus were calculated and statistically analyzed. The overall diagnostic accuracy for cancer was evaluated. RESULTS: The average scanning time to obtain the LCM image of a specimen was 1.6 seconds. The LCM images acquired corresponded well to the conventional H&E light microscopic images in the esophagus, stomach, and colon. Cell wall, nucleus, cytoplasm, and tissue structural elements were simultaneously visualized by LCM scanning. A difference in N/C ratios between normal mucosa and cancer in the esophagus was statistically apparent when Welch's test (P=0.05) was applied. The overall diagnostic accuracy of the LCM study for cancer was 89.7%. CONCLUSIONS: This novel method enables us to obtain an immediate serial virtual microscopic section through a fresh specimen, which has not actually been cut, although the resolution of the image obtained is still limited. These early results encourage us to develop imaging relevant to conventional histopathology alongside the development of LCM technology in the near future. We should aim at the in vivo application of LCM coupled to probes which can be introduced through the working channel of endoscopes.     

A hybrid network for automatic hepatocellular carcinoma segmentation in H&E-stained whole slide images

Hepatocellular carcinoma (HCC), as the most common type of primary malignant liver cancer, has become a leading cause of cancer deaths in recent years. Accurate segmentation of HCC lesions is critical for tumor load assessment, surgery planning, and postoperative examination. As the appearance of HCC lesions varies greatly across patients, traditional manual segmentation is a very tedious and time-consuming process, the accuracy of which is also difficult to ensure. Therefore, a fully automated and reliable HCC segmentation system is in high demand. In this work, we present a novel hybrid neural network based on multi-task learning and ensemble learning techniques for accurate HCC segmentation of hematoxylin and eosin (H&E)-stained whole slide images (WSIs). First, three task-specific branches are integrated to enlarge the feature space, based on which the network is able to learn more general features and thus reduce the risk of overfitting. Second, an ensemble learning scheme is leveraged to perform feature aggregation, in which selective kernel modules (SKMs) and spatial and channel-wise squeeze-and-excitation modules (scSEMs) are adopted for capturing the features from different spaces and scales. Our proposed method achieves state-of-the-art performance on three publicly available datasets, with segmentation accuracies of 0.797, 0.923, and 0.765 in the PAIP, CRAG, and UHCMC&CWRU datasets, respectively, which demonstrates its effectiveness in addressing the HCC segmentation problem. To the best of our knowledge, this is also the first work on the pixel-wise HCC segmentation of H&E-stained WSIs.     

DeepSMILE: Contrastive self-supervised pre-training benefits MSI and HRD classification directly from H&E whole-slide images in colorectal and breast cancer

We propose a Deep learning-based weak label learning method for analyzing whole slide images (WSIs) of Hematoxylin and Eosin (H&E) stained tumor tissue not requiring pixel-level or tile-level annotations using Self-supervised pre-training and heterogeneity-aware deep Multiple Instance LEarning (DeepSMILE). We apply DeepSMILE to the task of Homologous recombination deficiency (HRD) and microsatellite instability (MSI) prediction. We utilize contrastive self-supervised learning to pre-train a feature extractor on histopathology tiles of cancer tissue. Additionally, we use variability-aware deep multiple instance learning to learn the tile feature aggregation function while modeling tumor heterogeneity. For MSI prediction in a tumor-annotated and color normalized subset of TCGA-CRC (n=360 patients), contrastive self-supervised learning improves the tile supervision baseline from 0.77 to 0.87 AUROC, on par with our proposed DeepSMILE method. On TCGA-BC (n=1041 patients) without any manual annotations, DeepSMILE improves HRD classification performance from 0.77 to 0.81 AUROC compared to tile supervision with either a self-supervised or ImageNet pre-trained feature extractor. Our proposed methods reach the baseline performance using only 40% of the labeled data on both datasets. These improvements suggest we can use standard self-supervised learning techniques combined with multiple instance learning in the histopathology domain to improve genomic label classification performance with fewer labeled data.     

Integrated local binary pattern texture features for classification of breast tissue imaged by optical coherence microscopy

This paper proposes a texture analysis technique that can effectively classify different types of human breast tissue imaged by Optical Coherence Microscopy (OCM). OCM is an emerging imaging modality for rapid tissue screening and has the potential to provide high resolution microscopic images that approach those of histology. OCM images, acquired without tissue staining, however, pose unique challenges to image analysis and pattern classification. We examined multiple types of texture features and found Local Binary Pattern (LBP) features to perform better in classifying tissues imaged by OCM. In order to improve classification accuracy, we propose novel variants of LBP features, namely average LBP (ALBP) and block based LBP (BLBP). Compared with the classic LBP feature, ALBP and BLBP features provide an enhanced encoding of the texture structure in a local neighborhood by looking at intensity differences among neighboring pixels and among certain blocks of pixels in the neighborhood. Fourty-six freshly excised human breast tissue samples, including 27 benign (e.g. fibroadenoma, fibrocystic disease and usual ductal hyperplasia) and 19 breast carcinoma (e.g. invasive ductal carcinoma, ductal carcinoma in situ and lobular carcinoma in situ) were imaged with large field OCM with an imaging area of 10 × 10 mm² (10, 000 × 10, 000 pixels) for each sample. Corresponding H&E histology was obtained for each sample and used to provide ground truth diagnosis. 4310 small OCM image blocks (500 × 500 pixels) each paired with corresponding H&E histology was extracted from large-field OCM images and labeled with one of the five different classes: adipose tissue (n = 347), fibrous stroma (n = 2,065), breast lobules (n = 199), carcinomas (pooled from all sub-types, n = 1,127), and background (regions outside of the specimens, n = 572). Our experiments show that by integrating a selected set of LBP and the two new variant (ALBP and BLBP) features at multiple scales, the classification accuracy increased from 81.7% (using LBP features alone) to 93.8% using a neural network classifier. The integrated feature was also used to classify large-field OCM images for tumor detection. A receiver operating characteristic (ROC) curve was obtained with an area under the curve value of 0.959. A sensitivity level of 100% and specificity level of 85.2% was achieved to differentiate benign from malignant samples. Several other experiments also demonstrate the complementary nature of LBP and the two variants (ALBP and BLBP features) and the significance of integrating these texture features for classification. Using features from multiple scales and performing feature selection are also effective mechanisms to improve accuracy while maintaining computational efficiency.     

CS-CO: A Hybrid Self-Supervised Visual Representation Learning Method for H&E-stained Histopathological Images

Visual representation extraction is a fundamental problem in the field of computational histopathology. Considering the powerful representation capacity of deep learning and the scarcity of annotations, self-supervised learning has emerged as a promising approach to extract effective visual representations from unlabeled histopathological images. Although a few self-supervised learning methods have been specifically proposed for histopathological images, most of them suffer from certain defects that may hurt the versatility or representation capacity. In this work, we propose CS-CO, a hybrid self-supervised visual representation learning method tailored for H&E-stained histopathological images, which integrates advantages of both generative and discriminative approaches. The proposed method consists of two self-supervised learning stages: cross-stain prediction (CS) and contrastive learning (CO). In addition, a novel data augmentation approach named stain vector perturbation is specifically proposed to facilitate contrastive learning. Our CS-CO makes good use of domain-specific knowledge and requires no side information, which means good rationality and versatility. We evaluate and analyze the proposed CS-CO on three H&E-stained histopathological image datasets with downstream tasks of patch-level tissue classification and slide-level cancer prognosis and subtyping. Experimental results demonstrate the effectiveness and robustness of the proposed CS-CO on common computational histopathology tasks. Furthermore, we also conduct ablation studies and prove that cross-staining prediction and contrastive learning in our CS-CO can complement and enhance each other. Our code is made available at https://github.com/easonyang1996/CS-CO.     

PTEN and VEGF: possible predictors for sentinel lymph node micro-metastasis in breast cancer.

INTRODUCTION: The loss of PTEN expression and VEGF overexpression has been found to be correlated with metastasis in breast cancer patients. Despite significant advances in micro-metastasis detection methods, little is known about the relationship between micro-metastasis and primary tumors. The purpose of this study was to assess the association of VEGF and PTEN expression with micro-metastasis in breast cancer. MATERIALS AND METHODS: As destination sites for micro-metastasis, we examined peripheral blood (BD), bone marrow (BM) and sentinel lymph node (SLN) from 53 breast cancer patients. Protein and gene expressions of VEGF and PTEN at the primary site were determined by immunohistochemistry (IHC). BD and BM samples were processed using immunocytochemistry (ICC). SLNs were examined by hematoxylin and eosin (H&E) staining and IHC. RESULTS: Percentages of the patients with micro-metastasis were 24.5% for BD, 56.6% for BM, 26.4% in SLN by H&E and 41.5% in SLN by IHC. VEGF overexpression was strongly correlated to loss of PTEN expression (P=0.001, r=-0.446). VEGF overexpression and loss of PTEN expression were significantly associated with SLN micro-metastasis by either H&E or IHC (P<0.001). On the contrary, there is no significant correlation between their expression and micro-metastasis in BD and BM. CONCLUSIONS: Our results indicate possible value of using these biological markers to predict the risk of micro-metastasis in breast cancer.     

Joint fully convolutional and graph convolutional networks for weakly-supervised segmentation of pathology images

Tissue/region segmentation of pathology images is essential for quantitative analysis in digital pathology. Previous studies usually require full supervision (e.g., pixel-level annotation) which is challenging to acquire. In this paper, we propose a weakly-supervised model using joint Fully convolutional and Graph convolutional Networks (FGNet) for automated segmentation of pathology images. Instead of using pixel-wise annotations as supervision, we employ an image-level label (i.e., foreground proportion) as weakly-supervised information for training a unified convolutional model. Our FGNet consists of a feature extraction module (with a fully convolutional network) and a classification module (with a graph convolutional network). These two modules are connected via a dynamic superpixel operation, making the joint training possible. To achieve robust segmentation performance, we propose to use mutable numbers of superpixels for both training and inference. Besides, to achieve strict supervision, we employ an uncertainty range constraint in FGNet to reduce the negative effect of inaccurate image-level annotations. Compared with fully-supervised methods, the proposed FGNet achieves competitive segmentation results on three pathology image datasets (i.e., HER2, KI67, and H&E) for cancer region segmentation, suggesting the effectiveness of our method. The code is made publicly available at https://github.com/zhangjun001/FGNet.     

Deep triplet hashing network for case-based medical image retrieval

Deep hashing methods have been shown to be the most efficient approximate nearest neighbor search techniques for large-scale image retrieval. However, existing deep hashing methods have a poor small-sample ranking performance for case-based medical image retrieval. The top-ranked images in the returned query results may be as a different class than the query image. This ranking problem is caused by classification, regions of interest (ROI), and small-sample information loss in the hashing space. To address the ranking problem, we propose an end-to-end framework, called Attention-based Triplet Hashing (ATH) network, to learn low-dimensional hash codes that preserve the classification, ROI, and small-sample information. We embed a spatial-attention module into the network structure of our ATH to focus on ROI information. The spatial-attention module aggregates the spatial information of feature maps by utilizing max-pooling, element-wise maximum, and element-wise mean operations jointly along the channel axis. To highlight the essential role of classification in direntiating case-based medical images, we propose a novel triplet cross-entropy loss to achieve maximal class-separability and maximal hash code-discriminability simultaneously during model training. The triplet cross-entropy loss can help to map the classification information of images and similarity between images into the hash codes. Moreover, by adopting triplet labels during model training, we can utilize the small-sample information fully to alleviate the imbalanced-sample problem. Extensive experiments on two case-based medical datasets demonstrate that our proposed ATH can further improve the retrieval performance compared to the state-of-the-art deep hashing methods and boost the ranking performance for small samples. Compared to the other loss methods, the triplet cross-entropy loss can enhance the classification performance and hash code-discriminability.     

Brain white matter morphological structure correlation with its optical properties estimated from optical coherence tomography (OCT) data

A pilot post-mortem study identifies a strong correlation between the attenuation coefficient estimated from the OCT data and some morphological features of the sample, namely the number of nuclei in the field of view of the histological image and the fiber structural parameter introduced in the study to quantify the difference in the myelinated fibers arrangements. The morphological features were identified from the histopathological images of the sample taken from the same locations as the OCT images and stained with the immunohistochemical (IHC) staining specific to the myelin. It was shown that the linear regression of the IHC quantitative characteristics allows adequate prediction of the attenuation coefficient of the sample. This discovery opens the opportunity for the usage of the OCT as a neuronavigation tool.     

DDTNet: A dense dual-task network for tumor-infiltrating lymphocyte detection and segmentation in histopathological images of breast cancer

The morphological evaluation of tumor-infiltrating lymphocytes (TILs) in hematoxylin and eosin (H& E)-stained histopathological images is the key to breast cancer (BCa) diagnosis, prognosis, and therapeutic response prediction. For now, the qualitative assessment of TILs is carried out by pathologists, and computer-aided automatic lymphocyte measurement is still a great challenge because of the small size and complex distribution of lymphocytes. In this paper, we propose a novel dense dual-task network (DDTNet) to simultaneously achieve automatic TIL detection and segmentation in histopathological images. DDTNet consists of a backbone network (i.e., feature pyramid network) for extracting multi-scale morphological characteristics of TILs, a detection module for the localization of TIL centers, and a segmentation module for the delineation of TIL boundaries, where a boundary-aware branch is further used to provide a shape prior to segmentation. An effective feature fusion strategy is utilized to introduce multi-scale features with lymphocyte location information from highly correlated branches for precise segmentation. Experiments on three independent lymphocyte datasets of BCa demonstrate that DDTNet outperforms other advanced methods in detection and segmentation metrics. As part of this work, we also propose a semi-automatic method (TILAnno) to generate high-quality boundary annotations for TILs in H& E-stained histopathological images. TILAnno is used to produce a new lymphocyte dataset that contains 5029 annotated lymphocyte boundaries, which have been released to facilitate computational histopathology in the future.     

Do bcl‐2 and survivin help distinguish benign from malignant B‐cell lymphoid aggregates in bone marrow biopsies?

Bcl-2 and survivin are cellular proteins that are known to be inhibitors of apoptosis and are commonly found in malignant tissues, including lymphomas. In previous studies, it has been shown that staining for bcl-2 can help distinguish between benign and malignant lymphoid aggregates in bone marrow biopsies. To determine whether staining for survivin expression in lymphoid aggregates can aid investigators in making this clinically important distinction, we stained bone marrow biopsies from 10 patients with benign lymphoid aggregates, and 15 malignant ones derived from B cells (six mantle cell, four follicular cells, two diffuse large cell, two small lymphocytic cell, and one marginal zone lymphoma) with antibodies to CD3, CD20, bcl-2, and survivin by an indirect immunoperoxidase technique. Whereas staining for bcl-2 was significantly stronger in the malignant lymphoid aggregates (P=0.001), both the control and malignant cases were almost uniformly negative for survivin expression. Only three cases (two mantle cell and one small lymphocytic lymphoma) showed very faint expression of survivin. Although bcl-2 and survivin both act to inhibit apoptosis, their expressions do not parallel each other. Survivin is not significantly expressed in either benign or malignant bone marrow aggregates, and therefore measuring its expression does not help distinguish benign from malignant B-cell bone marrow lymphoid aggregates.     

Classification of follicular lymphoma images: A holistic approach with symbol-based machine learning methods

It is not very often to see a symbol-based machine learning approach to be used for the purpose of image classification and recognition. In this paper we will present such an approach, which we first used on the follicular lymphoma images. Lymphoma is a broad term encompassing a variety of cancers of the lymphatic system. Lymphoma is differentiated by the type of cell that multiplies and how the cancer presents itself. It is very important to get an exact diagnosis regarding lymphoma and to determine the treatments that will be most effective for the patient's condition. Our work was focused on the identification of lymphomas by finding follicles in microscopy images provided by the Laboratory of Pathology in the University Hospital of Tenerife, Spain. We divided our work in two stages: in the first stage we did image pre-processing and feature extraction, and in the second stage we used different symbolic machine learning approaches for pixel classification. Symbolic machine learning approaches are often neglected when looking for image analysis tools. They are not only known for a very appropriate knowledge representation, but also claimed to lack computational power. The results we got are very promising and show that symbolic approaches can be successful in image analysis applications.     

Case-based teaching and learning experiences

Case-based teaching and learning strategies can be utilized to assist advanced practice psychiatric nursing students in both obtaining requisite knowledge and enhancing clinical reasoning skills. We discuss the benefits of case-based learning in terms of how it (1) contributes to students' appropriate organization of information to be recalled later for use in clinical reasoning situations; (2) generates experiences that students would not otherwise have; (3) increases the visibility of students' clinical reasoning processes; and (4) enhances students' confidence. This article also explores three examples of case-based teaching and learning: use of written cases in a seminar; use of standardized patients in an assessment course; and utilization of web-based cases for learning assessment and intervention skills. Finally, we compare and contrast each of these methods in terms of their relative effectiveness in achieving each of the benefits.     

滤泡树突状细胞表达模式变化 在淋巴瘤病理鉴别诊断中的应用价值

目的 探讨滤泡树突状细胞(follicular dendritic cell,FDC)网在各种类型淋巴瘤中的形态变化模式及其在鉴别诊断中的临床应用价值。方法 采用免疫组化方法对56例各种类型的淋巴瘤进行CD21免疫组化染色,观察FDC网的形态变化模式。其中包括弥漫大B细胞淋巴瘤8例、伯基特淋巴瘤2例、小淋巴细胞性淋巴瘤6例、浆细胞瘤6例、MALT淋巴瘤3例、外周T细胞淋巴瘤6例、间变性大细胞淋巴瘤5例、NK/T细胞淋巴瘤8例、滤泡性淋巴瘤4例、套细胞淋巴瘤3例、AITL 3例、FDC肉瘤2例。结果 FDC网在各种类型淋巴瘤中的形态变化可归为4种模式:①破坏消减型:绝大部分淋巴瘤FDC网完全或部分破坏,包括弥漫大B细胞淋巴瘤、伯基特淋巴瘤、小淋巴细胞性淋巴瘤、浆细胞瘤、外周T细胞淋巴瘤、间变性大细胞淋巴瘤、NK/T细胞淋巴瘤; ②存在型:FDC网存在,甚至有数量增多,包括MALT淋巴瘤、滤泡性淋巴瘤、套细胞淋巴瘤; ③增生紊乱型:FDC网增多、变形、紊乱,如AITL; ④全表达型:FDC网在肿瘤组织中每个细胞表达,如FDC肉瘤。结论 FDC在各种类型淋巴瘤中存在不同的形态变化模式,在淋巴瘤鉴别诊断中具有重要的临床应用价值。     

Wide-field, high-resolution two-photon tissue mapping of human skin ex vivo

Background and objectives

Two-photon microscopy (TPM) generates high definition images from biological samples such as human skin in a non-invasive way. The focus of an ultrashort pulsed laser is scanned through the tissue and the fluorescence photons induced by two-photon excitation of endogeneous fluorophores are collected, measured and displayed as a scanning image. So far, one of the main disadvantages of TPM is the limitation of the scanning area to tenths of a millimeter by the field of view of the focusing optics. Essential pieces of information, such as the differentiation between benign or malign changes in biological tissue, are therefore not captured. Within a BMBF-supported research project “FluoTOM”, a new modified TPM scanning technique was developed to overcome these shortcomings, and pilot tests have been made on benign and malignant samples of human skin ex vivo.

Material and methods

The field of view of the new TPM was extended from the current standard of <500 μm to >5 mm. The spectral band option allows the synchronous recording of one to four images with exact local congruence, using photons from a choice of different spectral windows. These images are finally merged into a false-color presentation for interpretation. The spectral wavelengths can be chosen in the range of 350–650 nm according to a specific diagnostic question. Real-time photography is used as an adjunctive method to locate the scan position with respect to the sample to be examined. Samples under investigation were unstained tissue slides or full paraffin-embedded biopsy/tissue blocks. The images obtained by our new method were subsequently compared with standard H&E histology.

Results

Vertical laser scan images of superior quality from unstained histological slides as well as from the paraffin blocks of different benign and malignant cutaneous samples were made. When directly compared to the routine H&E histology, which is the gold standard, relevant tissue structures and changes in the normal architecture could be clearly identified and confirmed respectively.

Conclusion

A new non-invasive TPM scanning technique has been established that provides high-resolution imaging of biological tissue, such as human skin, in a distortion-free way and with a diagnostic relevant imaging size, thus allowing the assessment of tissue-significant medical issues in greater detail. Current use of our new method ex vivo shows that it is particularly appropriate for diagnosis and research applications of human skin.     

Transformer-based unsupervised contrastive learning for histopathological image classification

A large-scale and well-annotated dataset is a key factor for the success of deep learning in medical image analysis. However, assembling such large annotations is very challenging, especially for histopathological images with unique characteristics (e.g., gigapixel image size, multiple cancer types, and wide staining variations). To alleviate this issue, self-supervised learning (SSL) could be a promising solution that relies only on unlabeled data to generate informative representations and generalizes well to various downstream tasks even with limited annotations. In this work, we propose a novel SSL strategy called semantically-relevant contrastive learning (SRCL), which compares relevance between instances to mine more positive pairs. Compared to the two views from an instance in traditional contrastive learning, our SRCL aligns multiple positive instances with similar visual concepts, which increases the diversity of positives and then results in more informative representations. We employ a hybrid model (CTransPath) as the backbone, which is designed by integrating a convolutional neural network (CNN) and a multi-scale Swin Transformer architecture. The CTransPath is pretrained on massively unlabeled histopathological images that could serve as a collaborative local–global feature extractor to learn universal feature representations more suitable for tasks in the histopathology image domain. The effectiveness of our SRCL-pretrained CTransPath is investigated on five types of downstream tasks (patch retrieval, patch classification, weakly-supervised whole-slide image classification, mitosis detection, and colorectal adenocarcinoma gland segmentation), covering nine public datasets. The results show that our SRCL-based visual representations not only achieve state-of-the-art performance in each dataset, but are also more robust and transferable than other SSL methods and ImageNet pretraining (both supervised and self-supervised methods). Our code and pretrained model are available at https://github.com/Xiyue-Wang/TransPath.