Hybrid Convolution Neural Network in Classification of Cancer in Histopathology Images

Cancer statistics in 2020 reveals that breast cancer is the most common form of cancer among women in India. One in 28 women is likely to develop breast cancer during their lifetime. The mortality rate is 1.6 to 1.7 times higher than maternal mortality rates. According to the US statistics, about 42,170 women in the US are expected to die in 2020 from breast cancer. The chance of survival can be increased through early and accurate diagnosis of cancer. The pathologists manually analyze the histopathology images using high-resolution microscopes to detect the mitotic cells. This is a time-consuming process because there is a minute difference between the normal and mitotic cells. To overcome these challenges, an automatic analysis and detection of breast cancer by using histopathology images play a vital role in prognosis. Earlier researchers used conventional image processing techniques for the detection of mitotic cells. These methods were found to be producing results with low accuracy and time-consuming. Therefore, several deep learning techniques were adopted by researchers to increase the accuracy and minimize the time. The hybrid deep learning model is proposed for selecting abstract features from the histopathology images. In the proposed approach, we have concatenated two different CNN architectures into a single model for effective classification of mitotic cells. Convolution neural network (CNN) automatically detects efficient features without human intervention and classifies cancerous and non-cancerous images using a hybrid fully connected network. It is a computationally efficient, very powerful, and efficient model for performing automatic feature extraction. It detects different phenotypic signatures of nuclei. In order to enhance the accuracy and computational efficiency, the histopathology images are preprocessed, segmented, and feature extracted through CNN and fed into a hybrid CNN for classification. The hybrid CNN is obtained by concatenating two CNN models; together, this is called model leveraging. Model averaging can be improved by weighting the contributions of each sub-model to the combined prediction by the expected performance of the sub-model. The proposed hybrid CNN architecture with data preprocessing with median filter and Otsu-based segmentation technique is trained using 50,000 images and tested using 50,000 images. It provides an overall accuracy of 98.9%.

     

A new automatic image analysis method for assessing estrogen receptors’ status in breast tissue specimens

Manual assessment of estrogen receptors′ (ER) status from breast tissue microscopy images is a subjective, time consuming and error prone process. Automatic image analysis methods offer the possibility to obtain consistent, objective and rapid diagnoses of histopathology specimens. In breast cancer biopsies immunohistochemically (IHC) stained for ER, cancer cell nuclei present a large variety in their characteristics that bring various difficulties for traditional image analysis methods. In this paper, we propose a new automatic method to perform both segmentation and classification of breast cell nuclei in order to give quantitative assessment and uniform indicators of IHC staining that will help pathologists in their diagnostic. Firstly, a color geometric active contour model incorporating a spatial fuzzy clustering algorithm is proposed to detect the contours of all cell nuclei in the image. Secondly, overlapping and touching nuclei are separated using an improved watershed algorithm based on a concave vertex graph. Finally, to identify positive and negative stained nuclei, all the segmented nuclei are classified into five categories according to their staining intensity and morphological features using a trained multilayer neural network combined with Fisher's linear discriminant preprocessing. The proposed method is tested on a large dataset containing several breast tissue images with different levels of malignancy. The experimental results show high agreement between the results of the method and ground-truth from the pathologist panel. Furthermore, a comparative study versus existing techniques is presented in order to demonstrate the efficiency and the superiority of the proposed method.     

基于滑动块的深度卷积神经网络乳腺X线摄影图像肿块分割算法

目的:提出一种基于滑动块的深度卷积神经网络局部分类、整图乳腺肿块分割的算法,为临床诊断提供有效的肿块形态特征。方法:首先通过区域生长算法和膨胀算法提取患者乳腺区域,并进行数据归一化操作。为了得到每一个像素位置上的诊断信息,在图像的对应位置中滑动提取肿块类及非肿块类图像块,根据卷积神经网络提取其中的纹理信息并对图像块进行分类。通过整合图像块的预测分类结果,进行由粗到细的肿块分割,获得乳腺整图中像素级别的肿块分割。结果:通过比较先进的深度卷积神经网络模型,本文算法滑动块分类结果DenseNet模型下准确率达到96.71%,乳腺X线摄影图像全图肿块分割结果F1-score最优为83.49%。结论:本算法可以分割出乳腺X线摄影图像中的肿块,为后续的乳腺病灶诊断提供可靠的基础。     

基于融合多网络深层卷积特征和稀疏双关系正则化方法的乳腺癌图像分类研究

乳腺癌是全球女性癌症死亡的主要原因之一。现有诊断方法主要是医生通过乳腺癌观察组织病理学图像进行判断,不仅费时费力,而且依赖医生的专业知识和经验,使得诊断效率无法令人满意。针对以上问题,设计基于组织学图像的深度学习框架,以提高乳腺癌诊断准确性,同时减少医生的工作量。开发一个基于多网络特征融合和稀疏双关系正则化学习的分类模型:首先,通过子图像裁剪和颜色增强进行乳腺癌图像预处理;其次,使用深度学习模型中典型的3种深度卷积神经网络(InceptionV3、ResNet-50和VGG-16),提取乳腺癌病理图像的多网络深层卷积特征并进行特征融合;最后,通过利用两种关系(“样本-样本”和“特征-特征”关系)和l_F正则化,提出一种有监督的双关系正则化学习方法进行特征降维,并使用支持向量机将乳腺癌病理图像区分为4类—正常、良性、原位癌和浸润性癌。实验中,通过使用ICIAR2018公共数据集中的400张乳腺癌病理图像进行验证,获得93%的分类准确性。融合多网络深层卷积特征可以有效地捕捉丰富的图像信息,而稀疏双关系正则化学习可以有效降低特征冗余并减少噪声干扰,有效地提高模型的分类性能。     

Using classification and regression trees,liquid‐based cytology and nuclear morphometry for the discrimination of endometrial lesions

‘The objective of this study is to investigate the potential of classification and regression trees (CARTs) in discriminating benign from malignant endometrial nuclei and lesions. The study was performed on 222 histologically confirmed liquid based cytological smears, specifically: 117 benign cases, 62 malignant cases and 43 hyperplasias with or without atypia. About 100 nuclei were measured from each case using an image analysis system; in total, we collected 22783 nuclei. The nuclei from 50% of the cases (the training set) were used to construct a CART model that was used for knowledge extraction. The nuclei from the remaining 50% of cases (test set) were used to evaluate the stability and performance of the CART on unknown data. Based on the results of the CART for nuclei classification, we propose two classification methods to discriminate benign from malignant cases. The CART model had an overall accuracy for the classification of endometrial nuclei equal to 85%, specificity 90.68%, and sensitivity 72.05%. Both methods for case classification had similar performance: overall accuracy in the range 94–95%, specificity 95%, and sensitivity 91–94%. The results of the proposed system outperform the standard cytological diagnosis of endometrial lesions. This study highlights interesting diagnostic features of endometrial nuclear morphology and provides a new classification approach for endometrial nuclei and cases. The proposed method can be a useful tool for the everyday practice of the cytological laboratory. Diagn. Cytopathol. 2014;42:582–591. © 2013 Wiley Periodicals, Inc.     

基于深度学习结合解剖学注意力机制的肺结节良恶性分类

肺结节作为肺癌的初期表现,及时的发现和准确的良恶性诊断对于疾病的治疗具有重要的意义。为了提高肺部CT图像中肺结节良恶性的诊断率,提出一种基于3D ResNet的卷积神经网络,并通过加入解剖学注意力模块有效地提高了肺结节良恶性的分类精度。此外,该方法通过自动分割以获取注意力机制所需的感兴趣区域,实现整个流程的全自动化。解剖学注意力的添加能更好地捕捉图像中的局部纹理信息,进一步提取对于肺结节良恶性诊断有用的特征。本文方法在LIDC-IDRI数据集上进行验证。实验结果表明与传统的3D ResNet及其他现有的方法相比,本文方法在分类精度上有显著的提高,在独立测试集上的最终分类的AUC达到0.973,准确率为0.940。由此可见,本文方法能在辅助医生对肺结节的诊断中起到重要作用。     

基于PCA和LDA方法的肿瘤基因表达谱数据分类

目的基因芯片技术对医学临床诊断、治疗、药物开发和筛选等技术的发展具有革命性的影响。针对高维医学数据降维困难及基因表达谱样本数据少、维度高、噪声大的特点,维数约减十分必要。基于主成分分析（principalcomponentanalysis,PCA）和线性判别分析（1ineardiscriminantanalysis,LDA）方法,有效解决了基因表达谱数据分类问题,并提高了识别率。方法分别引人PCA和LDA方法对基因表达谱数据进行降维,然后用K近邻（K—nearestneighbor,KNN）作为分类器对数据进行分类,并分别在乳腺癌和卵巢癌质谱数据上。结果在两类癌症质谱数据上应用PCA和LDA方法能够有效提取分类特征信息,并在保持较高分类正确率的前提下大幅度降低医学数据的维数。结论利用维数约减的方法对癌症基因表达谱数据进行分类,可辅助临床医生发现新的疾病特征,提高疾病诊断的正确率。     

Nuclear planimetry on the samples from cytologically overlapping proliferative breast lesions

Differentiation of low-grade breast carcinomas from fibroadenomas with atypia on fine-needle aspiration cytology material may sometimes be problematic. In some cytological samples of these lesions, both the nuclei of cells and patterns of cell clusters display substantial overlapping morphological characteristics. Nuclear morphometry on cytological material is suggested as an ancillary method for the differential diagnosis in many lesions, including breast tumors. Twenty-five cytological samples obtained from patients with breast lesions, which were histopathologically confirmed as grade I ductal carcinomas (n=10), tubular carcinomas (n=5), and fibroadenomas with atypia (n=10), were utilized in this study. Eight geometric features of about 2002 nuclei from these tumors were measured. Discriminant analysis was performed on this data set in order to test the correct classification based on the eight measured variables. Statistical analyses were carried out with two fundamentally different approaches: in the first one, the entire data from all measured nuclei were used for classification. In the second one, a subset of data representing the 10% most deviated values of variables was extracted from the entire dataset to simulate the “selective examination” performed during classical morphologic evaluation. When the entire data was used in discriminant analysis, the overall performance in the correct classification rate was found to be approximately 50%, which was considered an unacceptable value in routine diagnostic practice. In the subset of data constructed with our systematic and reproducible “selection bias”, the overall performance of correct classification rate of the same discriminant model improved substantially to 97%. Morphologic examination is actually based on selection. The use of data obtained from all of the cells in morphometry, as previously used in nearly all of the statistical methods, may cause a masking effect in diagnostically important features. Morphometric studies may seem to be useless when this effect is not taken into account. However, with a systematic and reproducible selection of the values with a proper “bias”, morphometry may provide some discriminatory information in overlapping lesions.     

基于定量影像组学的乳腺肿瘤良恶性诊断

乳腺癌是女性致死率最高的恶性肿瘤之一。为提高诊断效率,提供给医生更加客观和准确的诊断结果。借助影像组学的方法,利用公开数据集BreaKHis中82例患者的乳腺肿瘤病理图像,提取乳腺肿瘤病理图像的灰度特征、Haralick纹理特征、局部二值模式(LBP)特征和Gabor特征共139维影像组学特征,并用主成分分析(PCA)对影像组学特征进行降维,然后利用随机森林(RF)、极限学习机(ELM)、支持向量机(SVM)、k最近邻(kNN)等4种不同的分类器构建乳腺肿瘤良恶性的诊断模型,并对上述不同的特征集进行评估。结果表明,基于支持向量机的影像组学特征的分类效果最好,准确率能达到88.2%,灵敏性达到86.62%,特异性达到89.82%。影像组学方法可为乳腺肿瘤良恶性预测提供一种新型的检测手段,使乳腺肿瘤良恶性临床诊断的准确率得到很大提升。     

Prognostic value of nucleolar morphometric variables in cytological breast cancer specimens.

In a retrospective study on cytological specimens from 86 patients with histologically confirmed invasive breast cancer, the prognostic value of nucleolar morphometric variables was studied and compared with nuclear variables. One hundred nuclei and their nucleoli on each slide were measured with a graphic tablet system at a total magnification of 2800 times using a stratified selection method. The number of nucleoli per 100 nuclei was also noted. Analysis of Kaplan-Meier univariate recurrence free survival curves showed significant differences for eight nuclear features, nine nucleolar features, and three combined nuclear and nucleolar variables. The total number of nucleoli per 100 nuclei was the best single prognostic variable. Multivariate survival analysis (Cox regression model) showed that no other features provided additional prognostic information beyond that given by the total number of nucleoli. It is concluded that nucleolar morphometric variables assessed in cytological preparations have prognostic value in breast cancer, and the results of this study suggest that their prognostic value may exceed that of nuclear variables.     

基于PET/CT图像纹理参数的肺结节诊断模型

目的基于PET/CT融合图像纹理参数建立肺结节良恶性诊断模型,提高肺癌的识别率。方法选取宣武医院核医学科经PET/CT检查的52例肺结节患者,收集其PET/CT影像图像及人口学、影像学信息。以Contourlet变换和灰度共生矩阵相结合的方式,对PET/CT图像的感兴趣区域提取纹理参数。基于所提取的纹理参数建立支持向量机模型,得到每个肺结节良恶性判别结果。为了提高模型的诊断效果,将结节边缘、最大摄取值、有晕征等影像学信息也纳入模型,重新建立支持向量机模型。通过灵敏度、特异度、正确率等指标对模型诊断效果进行评价。结果纹理参数肺结节诊断模型的灵敏度、特异度分别为90.7%、93.5%,纹理参数结合影像学信息的肺结节诊断模型的灵敏度、特异度分别为95.7%、100.0%。结论基于PET/CT图像纹理参数建立的支持向量机模型对良恶性肺结节具有较好的鉴别诊断效果。     

Comparison of light microscopic grading and morphometric features in cytological breast cancer specimens

A study was set up to investigate correlations between different light microscopic grades and morphometric features in cytological breast cancer specimens and to evaluate the discriminative power of morphometry for the three grades. 76 slides (smears and imprints) were graded independently in two different laboratories, and the 54 unequivocally graded slides were used as a training set for computing classification rules. In all slides, 100 nuclei and their nucleoli were measured on a graphic tablet at a final magnification of 2800x. A total of 43 morphometric features was evaluated. Univariate analysis showed significantly different values for most of the morphometric features for the three grades. Discriminant analysis revealed that a combination of the mean nuclear area and the number of mitoses per slide provided optimal discrimination between grades one and two (87.5% correct classification). One more feature (mean nuclear shape factor) was needed to obtain optimal discrimination between grades two and three (83.3% correct classifications). 16 of the 22 (72.7%) equivocally graded slides could be classified with high probability using these classification rules. We conclude that morphometry has discriminative power for the three grades in cytologic breast cancer specimens and may therefore be a useful instrument for classification of difficult cases.     

基于自适应能量偏移场无边缘主动轮廓模型的乳腺肿块分割与分类方法研究

目的:为提高乳腺癌检测的精准度和效率,提出了一种基于自适应能量偏移场无边缘主动轮廓模型（AEOF-CV）的乳腺肿块分割与分类方法。方法:首先采用中值滤波、阈值分割及区域连通进行图像预处理,去除图像噪声;然后使用伽马变换及形态学运算相结合的方法进行图像增强;其次,采用AEOF-CV对弱对比度图像提高分割精度,用于乳腺肿块分割,得到感兴趣区域;最后使用不同提取特征方法,结合支持向量机识别感兴趣区域是否有肿块,并对存在肿块的图像判别肿块的良、恶性。结果:实验利用DDSM数据库中350个图像进行测试,实验结果证明,基于AEOF-CV乳腺肿块分割方法可以得到肿块清晰外部轮廓,具有较好的鲁棒性,误分率可达到0.212 0。无肿块样本识别率达到94.57%,恶性肿块识别率为97.91%,良性肿块识别率为96.96%,总识别率达94.00%。结论:基于AEOF-CV的乳腺肿块分割效果较好,误分率相对CV方法降低19.17%,查准率和查全率达到了0.851 9和0.836 5,全局分析性能较好,是乳腺肿块分割的有效方法,可为后续模式识别提供可靠依据。     

Fine-needle aspiration cytology of lobular carcinoma in situ

Fine-needle aspiration cytology (FNAC) plays a key role in the preoperative diagnosis of breast carcinoma but is less reliable in the diagnosis of in situ lesions. The objective of the present study was to investigate the cytological features of lobular carcinoma in situ (LCIS), regarding which little data is available to date. Cytological features of FNAC of the breast from 21 patients with histology-proven LCIS were described and compared with surgical specimens. Aspirates from 8/21 cases had cell groups diagnostic for or compatible with LCIS. Aspirates from an additional two cases demonstrated hypercellular, dissociated, and more pleomorphic tumor cells, which were originally diagnosed as invasive lobular carcinoma (ILC). The remaining 11 aspirates were diagnosed as benign or nondiagnostic. FNAC from the eight diagnostic specimens were characterized by loosely cohesive cell groups composed of uniform cells with occasional intracytoplasmic lumina, slightly irregular and eccentric nuclei. We conclude that the main difficulty in diagnosing LCIS by FNAC is sampling rather than recognition of the lesions. However, one should be aware of the cytological features of LCIS in order to reach a correct diagnosis. There are no reliable cytological criteria that help in differentiating pleomorphic and dissociated LCIS from ILC.     

Cell segmentation in histopathological images with deep learning algorithms by utilizing spatial relationships

In many computerized methods for cell detection, segmentation, and classification in digital histopathology that have recently emerged, the task of cell segmentation remains a chief problem for image processing in designing computer-aided diagnosis (CAD) systems. In research and diagnostic studies on cancer, pathologists can use CAD systems as second readers to analyze high-resolution histopathological images. Since cell detection and segmentation are critical for cancer grade assessments, cellular and extracellular structures should primarily be extracted from histopathological images. In response, we sought to identify a useful cell segmentation approach with histopathological images that uses not only prominent deep learning algorithms (i.e., convolutional neural networks, stacked autoencoders, and deep belief networks), but also spatial relationships, information of which is critical for achieving better cell segmentation results. To that end, we collected cellular and extracellular samples from histopathological images by windowing in small patches with various sizes. In experiments, the segmentation accuracies of the methods used improved as the window sizes increased due to the addition of local spatial and contextual information. Once we compared the effects of training sample size and influence of window size, results revealed that the deep learning algorithms, especially convolutional neural networks and partly stacked autoencoders, performed better than conventional methods in cell segmentation.     

Delineation of the ischemic stroke lesion based on watershed and relative fuzzy connectedness in brain MRI

Precise segmentation of stroke lesions from brain magnetic resonance (MR) images poses a challenging task in automated diagnosis. In this paper, we proposed a new method called watershed-based lesion segmentation algorithm (WLSA), which is a novel intensity-based segmentation technique used to delineate infarct lesion in diffusion-weighted imaging (DWI) MR images of the brain. The algorithm was tested on a series of 142 real-time images collected from different stroke patients reported at IMS and SUM Hospital. One MRI slice having largest area of infract lesion is selected from each patient from multiple slices. The main objective is to combine the strength of guided filter and watershed transform through relative fuzzy connectedness (RFC) to detect lesion boundaries appropriately. The extracted informative statistical and geometrical features are used to classify the types of stroke lesions according to the Oxfordshire Community Stroke Project (OCSP) classification. The experimental results demonstrated the effectiveness of the proposed process with high accuracy in delineating lesions. A classification with a dice similarity index (DSI) of 96% with computational time of 0.06 s in random forest (RF) and an accuracy of 85% with computational time of 0.84 s has been obtained by multilayer perceptron (MLP) neural network classifier in tenfold cross-validation process. Better detection accuracy is achieved in RF classifier in classifying stroke lesions.     

Preclinical study of using multiphoton microscopy to diagnose liver cancer and differentiate benign and malignant liver lesions

Recently, the miniaturized multiphoton microscopy (MPM) and multiphoton probe allow the clinical use of multiphoton endoscopy for diagnosing cancer via "optical biopsy". The purpose of this study was to establish MPM optical diagnostic features for liver cancer and evaluate the sensitivity, specificity, and accuracy of MPM optical diagnosis. Firstly, we performed a pilot study to establish the MPM diagnostic features by investigating 60 surgical specimens, and found that high-resolution MPM images clearly demonstrated apparent differences between benign and malignant liver lesions in terms of their tissue architecture and cell morphology. Cancer cells, characterized by irregular size and shape, enlarged nuclei, and increased nuclear-cytoplasmic ratio, were identified by MPM images, which were comparable to hematoxylin-eosin staining images. Secondly, we performed a blinded study to evaluate the sensitivity, specificity, and accuracy of MPM optical diagnosis by investigating another 164 specimens, and found that the sensitivity, specificity, and accuracy of MPM diagnosis was 96.32%, 96.43%, and 96.34%, respectively. In conclusion, it is feasible to use MPM to diagnose liver cancer and differentiate benign and malignant liver lesions. This preclinical study provides the groundwork for further using multiphoton endoscopy to perform real-time noninvasive "optical biopsy" for liver lesions in the near future.     

A statistical based feature extraction method for breast cancer diagnosis in digital mammogram using multiresolution representation

This paper presents a method for breast cancer diagnosis in digital mammogram images. Multi-resolution representations, wavelet or curvelet, are used to transform the mammogram images into a long vector of coefficients. A matrix is constructed by putting wavelet or curvelet coefficients of each image in row vector, where the number of rows is the number of images, and the number of columns is the number of coefficients. A feature extraction method is developed based on the statistical t-test method. The method is ranking the features (columns) according to its capability to differentiate the classes. Then, a dynamic threshold is applied to optimize the number of features, which can achieve the maximum classification accuracy rate. The method depends on extracting the features that can maximize the ability to discriminate between different classes. Thus, the dimensionality of data features is reduced and the classification accuracy rate is improved. Support vector machine (SVM) is used to classify between the normal and abnormal tissues and to distinguish between benign and malignant tumors. The proposed method is validated using 5-fold cross validation. The obtained classification accuracy rates demonstrate that the proposed method could contribute to the successful detection of breast cancer.     

基于空洞空间金字塔池化的U-Net网络在肺部图像分割上的应用

目的：胸部X线图像中肺野的自动分割是相关疾病筛查和诊断的关键步骤,为了适应计算机辅助诊断系统的要求,提出一种基于空洞空间金字塔池化的U-Net网络对胸部X线图像中肺野进行自动分割。方法：在编码和解码之间引入带有空洞卷积的空间金字塔池化用于扩大接受域;同时,在多个尺度上获取图像上下文信息,用于从胸片中分割肺野,使用Montgomery数据集及深圳数据集进行验证。根据医学图像分割常用指标准确性、Dice相似系数及交并比评价基于空洞空间金字塔池化的U-Net网络分割肺野的性能。结果：验证准确性为98.29%,Dice相似系数为96.61%,交并比为93.47%。结论：本文提出一种基于空洞空间金字塔池化的U-Net网络用于分割肺野,相较于其他方法学习到更多边缘分割特征,取得更好的分割结果。     

图像分割技术在中医舌诊客观化研究中的应用

舌诊是中医四诊的主要内容,是辨证论治的主要依据。客观化研究对中医辨证规范化及中医临床、教学和科研手段的现代化具有重要意义。对舌诊客观化研究中涉及的图像预处理的重要内容——舌体分割提取和舌苔舌质同类区域划分——进行了深入研究,提出了相应算法,通过实验充分证明了算法具有很好的鲁棒性。这给进一步的自动特征提取提供了保障和重要信息。