Integrated expert systems and videodisc in surgical pathology: an overview

We present an overview of our 6-year experience in the design of expert systems for anatomic pathology. Our practical goal is to help practicing pathologists with learning, teaching, and the task of diagnosis by providing them with dynamic expert knowledge by means of a personal computer. This project could only be undertaken by first addressing a scientific goal: to characterize the problem-solving strategies that expert pathologists use in making a diagnosis and to state them in the logical terms of computer science. Our approach has been to build systems first for experimentation and then for use. The result of our work is an integrated computer-based approach that handles expert knowledge as formal relationships and morphologic images and that uses a number of logical strategies to provide multiple perspectives on diagnostic tasks. Configured as a pathologist's workstation, this approach can be expected to enhance the performance of trained general pathologists and pathologists in training. Lymph node pathology has been used as the prototype domain for this research, but care has been taken to seek a generalized authoring and inference structure that can be applied to other areas of pathology by changing the contents but not the structure itself. Excursions into various surgical pathology specialties suggest that the ways the system is constructed and exercised is fundamentally robust. Such computer-based expert systems can be expected to generate a new standard in the practice of pathology--based on the "gold standard" of classical morphology, but including the coordinated use of new methods from immunology and molecular biology in a multidisciplinary approach to diagnosis when these techniques are relevant. The benefits from this technology can be expected to be widespread with the evolution, refinement, and diffusion of these systems.     

基于病理图像的计算机辅助诊断进展

临床上对癌症的组织病理诊断是所有诊断方式的金标准。由于病理医师的主观决策性,基于显微镜观察的诊断结果准确率不高。随着计算机技术的快速发展,计算机辅助诊断用于病理图像分析成为人工智能领域的潮流。本研究对近年来病理图像辅助诊断的相关文献进行回顾,重点论述病理图像来源、机器学习的分阶段处理、端到端的全自动诊断及病理图像检索等方面的研究进展,最后对基于病理图像的计算机辅助诊断的发展趋势进行展望。     

Cancerous nuclei detection on digitized pathological lung color images

In this paper, we propose a methodology (in the form of a software package) for automatic extraction of the cancerous nuclei in lung pathological color images. We first segment the images using an unsupervised Hopfield artificial neural network classifier and we label the segmented image based on chromaticity features and histogram analysis of the RGB color space components of the raw image. Then, we fill the holes inside the extracted nuclei regions based on the maximum drawable circle algorithm. All corrected nuclei regions are then classified into normal and cancerous using diagnostic rules formulated with respect to the rules used by experimented pathologist. The proposed method provides quantitative results in diagnosing a lung pathological image set of 16 cases that are comparable to an expert's diagnosis.     

Feature Selection and Performance Evaluation of Support Vector Machine (SVM)-Based Classifier for Differentiating Benign and Malignant Pulmonary Nodules by Computed Tomography

There are lots of work being done to develop computer-assisted diagnosis and detection (CAD) technologies and systems to improve the diagnostic quality for pulmonary nodules. Another way to improve accuracy of diagnosis on new images is to recall or find images with similar features from archived historical images which already have confirmed diagnostic results, and the content-based image retrieval (CBIR) technology has been proposed for this purpose. In this paper, we present a method to find and select texture features of solitary pulmonary nodules (SPNs) detected by computed tomography (CT) and evaluate the performance of support vector machine (SVM)-based classifiers in differentiating benign from malignant SPNs. Seventy-seven biopsy-confirmed CT cases of SPNs were included in this study. A total of 67 features were extracted by a feature extraction procedure, and around 25 features were finally selected after 300 genetic generations. We constructed the SVM-based classifier with the selected features and evaluated the performance of the classifier by comparing the classification results of the SVM-based classifier with six senior radiologists′ observations. The evaluation results not only showed that most of the selected features are characteristics frequently considered by radiologists and used in CAD analyses previously reported in classifying SPNs, but also indicated that some newly found features have important contribution in differentiating benign from malignant SPNs in SVM-based feature space. The results of this research can be used to build the highly efficient feature index of a CBIR system for CT images with pulmonary nodules.     

基于融合多网络深层卷积特征和稀疏双关系正则化方法的乳腺癌图像分类研究

乳腺癌是全球女性癌症死亡的主要原因之一。现有诊断方法主要是医生通过乳腺癌观察组织病理学图像进行判断,不仅费时费力,而且依赖医生的专业知识和经验,使得诊断效率无法令人满意。针对以上问题,设计基于组织学图像的深度学习框架,以提高乳腺癌诊断准确性,同时减少医生的工作量。开发一个基于多网络特征融合和稀疏双关系正则化学习的分类模型:首先,通过子图像裁剪和颜色增强进行乳腺癌图像预处理;其次,使用深度学习模型中典型的3种深度卷积神经网络(InceptionV3、ResNet-50和VGG-16),提取乳腺癌病理图像的多网络深层卷积特征并进行特征融合;最后,通过利用两种关系(“样本-样本”和“特征-特征”关系)和l_F正则化,提出一种有监督的双关系正则化学习方法进行特征降维,并使用支持向量机将乳腺癌病理图像区分为4类—正常、良性、原位癌和浸润性癌。实验中,通过使用ICIAR2018公共数据集中的400张乳腺癌病理图像进行验证,获得93%的分类准确性。融合多网络深层卷积特征可以有效地捕捉丰富的图像信息,而稀疏双关系正则化学习可以有效降低特征冗余并减少噪声干扰,有效地提高模型的分类性能。     

Progression in diagnostic pathology; development of virtual microscopy and its applications

Many systems have already been designed and successfully used for clinical laboratory and pathological examination. The evolution of image analysis was enabled when analog images of the original glass slides could be transferred to digital images with the rapid development of virtual microscopy and virtual slides depended upon computer technologies. Today, whole slide can be acquired by virtual microscopes. The applications of virtual microscopy and virtual slides for teaching, diagnosis, telepathology, and research are more widely used than those of real microscope and real glass slides. In traditional cancer diagnosis, pathologists examine biopsies to make diagnostic assessments largely based on two-dimensional cell morphology and tissue distribution. These assessments are subjective and often show considerable variability. However, automated cancer diagnostic system based on three-dimensional image analysis based on nuclear bulging sign enables objective judgments using quantitative measurements. We expect that the shortage of pathologists will be improved when an automated cancer diagnosis system is developed.     

Ultrastructural telepathology--remote EM-diagnostic via Internet

New collaboratory opportunities in ultrastructural research and diagnostics are now available on the Internet through the combination of digital image acquisition, remote operation of modern digitally controlled and automated electron microscopes, and the development of software specifically tailored for collaboratory needs. Remote experts can examine samples directly, and unique instruments can be utilized from anywhere. In the case of diagnostic dilemmas, the second-opinion expert is no longer constrained by problems inherent in the interpretation of preselected images. The remote examiner can independently choose the area of interest on the sample as well as select the appropriate magnification for an accurate diagnosis. With these capabilities and together with teleconferencing tools and securely accessible databases on-line, telepathology can provide increased effectiveness and support for diagnostics, research, and teaching in many areas. The authors report their experience with remote electron microscope diagnoses of pathological samples using two different dynamic imaging systems and discuss the main technical issues encountered. It appears that only minor technical issues need to be resolved before ultrastructural telepathology can be promoted for routine use in areas with high-speed Internet access.     

The practice of teleconferencing in clinical pathology--networking among cancer-center-hospitals in various parts of Japan

In the last several years, we have been able to use a telemedicine system on a network among cancer-center-hospitals connected by light fibers(so-called Cancer Network) and attend teleconferences from each site in Japan at the same time using the same images. Every week we have many medical conferences using this network system. On the practical use of teleconferences, the ratio of clinical pathological images is very high, especially histopathological appreciation is important; that is, surgical slide conference, clinico-pathological conference, orthopedic tumor conference, image conferences about digestive organs, presentation of current topics in laboratory medicine and other issues are carried out by clinicopathological images. At present, images on teleconference are still-pictures and images in High Definition Television are clear and high capacity, and of sufficient quality for pathological diagnosis. However, the coincidence-rates of histopathological diagnosis among 15 pathologists between the tele-image method and direct microscopic method varied from 38-80%. It is necessary to try to experience the images of still-picture and also animated cartoon. In the near future, the present network may extend to cover a wide areas and attend to teleconferences in every medical facilities. By attending this network system, we are able to use clinicopathological information for clinical diagnosis, treatment, research and education.     

Cell segmentation in histopathological images with deep learning algorithms by utilizing spatial relationships

In many computerized methods for cell detection, segmentation, and classification in digital histopathology that have recently emerged, the task of cell segmentation remains a chief problem for image processing in designing computer-aided diagnosis (CAD) systems. In research and diagnostic studies on cancer, pathologists can use CAD systems as second readers to analyze high-resolution histopathological images. Since cell detection and segmentation are critical for cancer grade assessments, cellular and extracellular structures should primarily be extracted from histopathological images. In response, we sought to identify a useful cell segmentation approach with histopathological images that uses not only prominent deep learning algorithms (i.e., convolutional neural networks, stacked autoencoders, and deep belief networks), but also spatial relationships, information of which is critical for achieving better cell segmentation results. To that end, we collected cellular and extracellular samples from histopathological images by windowing in small patches with various sizes. In experiments, the segmentation accuracies of the methods used improved as the window sizes increased due to the addition of local spatial and contextual information. Once we compared the effects of training sample size and influence of window size, results revealed that the deep learning algorithms, especially convolutional neural networks and partly stacked autoencoders, performed better than conventional methods in cell segmentation.     

Ultrastructural Telepathology—Remote EM-Diagnostic via Internet

New collaboratory opportunities in ultrastructural research and diagnostics are now available on the Internet through the combination of digital image acquisition, remote operation of modern digitally controlled and automated electron microscopes, and the development of software specifically tailored for collaboratory needs. Remote experts can examine samples directly, and unique instruments can be utilized from anywhere. In the case of diagnostic dilemmas, the second-opinion expert is no longer constrained by problems inherent in the interpretation of preselected images. The remote examiner can independently choose the area of interest on the sample as well as select the appropriate magnification for an accurate diagnosis. With these capabilities and together with teleconferencing tools and securely accessible databases on-line, telepathology can provide increased effectiveness and support for diagnostics, research, and teaching in many areas. The authors report their experience with remote electron microscope diagnoses of pathological samples using two different dynamic imaging systems and discuss the main technical issues encountered. It appears that only minor technical issues need to be resolved before ultrastructural telepathology can be promoted for routine use in areas with high-speed Internet access.     

医学图像识别中多分类器融合方法的研究进展

计算机辅助医学图像分析识别对多种疾病的临床诊断有着重要的意义。由于医学图像自身的复杂性,单一分类器的识别性能常常难以满足临床上的要求,因此近年来,作为一种能有效改进单一分类器识别性能的方法,多分类器融合技术被逐步应用到包括乳腺X光片识别、肿瘤细胞识别以及内窥镜图像分析等领域,并取得了更为满意的识别结果。在参阅大量文献的基础上,对多分类器融合识别技术的理论分析及其在医学领域的研究及应用现状进行了综述,进而对其存在的问题进行了分析以及前景展望。     

Long distance image transfer: first results of its use in histopathological diagnosis.

Histopathological images were transferred by use of normal telephone lines between three pathology institutes located in three different cities in the FRG. Images were digitized using a colour TV camera and stored in a special computerized image transmission system. The stored image was transferred and visualized on a (receiver) colour TV screen after dialing the telephone number connected to the receiver image transmission system. An additional telephone dialogue was activated by use of a normal acoustic telephone, and the diagnostic difficulties of the underlying image were discussed. Diagnostic assistance was possible in all transferred cases as well as histopathological diagnosis. Resolution of the images was set at 512 x 512 pixel x 8 bit. Image transfer time was 3.2 min on average. The differences between the original and transferred image were measured by "retransfer" of the original image and by subtracting the two images from each other. No major transfer errors could be measured.     

人工智能技术在医学影像专家系统中的应用及发展

本文对人工智能技术在常规医学及其医学影像专家系统中的发展情况作了回顾 ,阐明了在医学诊断系统中 ,主要困难在于多种疾病的同时并发 ,即在许多病人中存在着一种疾病的症状潜伏着多种其它疾病的症状的现象。而医学影像专家系统发展的困难在于高级视觉系统内在的不足 ,从医学扫描器上获得的数据可能是噪声和模糊的 ,从而增加了专家系统的复杂性。最后对人工智能技术在医学影像专家系统中的发展前景作了展望。     

Evaluation of whole slide image immunohistochemistry interpretation in challenging prostate needle biopsies

Whole slide images (WSIs), also known as virtual slides, can support electronic distribution of immunohistochemistry (IHC) stains to pathologists that rely on remote sites for these services. This may lead to improvement in turnaround times, reduction of courier costs, fewer errors in slide distribution, and automated image analyses. Although this approach is practiced de facto today in some large laboratories, there are no clinical validation studies on this approach. Our retrospective study evaluated the interpretation of IHC stains performed in difficult prostate biopsies using WSIs. The study included 30 foci with IHC stains identified by the original pathologist as both difficult and pivotal to the final diagnosis. WSIs were created from the glass slides using a scanning robot (T2, Aperio Technologies, Vista, CA). An evaluation form was designed to capture data in 2 phases: (1) interpretation of WSIs and (2) interpretation of glass slides. Data included stain interpretations, diagnoses, and other parameters such as time required to diagnose and image/slide quality. Data were also collected from an expert prostate pathologist, consensus meetings, and a poststudy focus group. WSI diagnostic validity (intraobserver pairwise kappa statistics) was "almost perfect" for 1 pathologist, "substantial" for 3 pathologists, and "moderate" for 1 pathologist. Diagnostic agreement between the final/consensus diagnoses of the group and those of the domain expert was "almost perfect" (kappa = 0.817). Except for one instance, WSI technology was not felt to be the cause of disagreements. These results are encouraging and compare favorably with other efforts to quantify diagnostic variability in surgical pathology. With thorough training, careful validation of specific applications, and regular postsignout review of glass IHC slides (eg, quality assurance review), WSI technology can be used for IHC stain interpretation.     

Application of a fuzzy pattern classifier to decision making in portal verification of radiotherapy

With the large volume of electronic portal images acquired and stringent time constraints, it is no longer feasible to follow the convention whereby the radiation oncologist reviews and approves or rejects all portals. For that purpose we have developed a portal image classifier based on the fuzzy k-nearest neighbour (k-NN) algorithm. Each portal image is represented by a feature vector that consists of translational and rotational errors in the placement of radiation field borders that were measured in the portal image. Memberships in the acceptable portal class for the reference portal images within a training dataset were defined by a radiation oncologist expert. The fuzzy k-NN portal image classifier was trained and tested on a dataset of 328 portal images acquired during tangential irradiations of the breast. The memberships in the acceptable portal class produced by the fuzzy k-NN algorithm agreed very well with those defined by the expert. The linear correlation coefficient was equal to 0.89. Performance of the fuzzy k-NN classifier was also evaluated from the portal decision-making point of view using the measures of accuracy, sensitivity and specificity. The fuzzy k-NN portal classifier was capable of identifying almost all the truly unacceptable portals with an acceptably low false alarm rate.     

Diagnosis and classification of psoriasis

Psoriasis is a chronic inflammatory multi organ disease with well characterized pathology occurring in the skin and often the joints. Although the disease has many characteristic and even pathognomonic features, no established diagnostic criteria exist for cutaneous psoriasis and there is no unified classification for the clinical spectrum of the disease. Prior approaches that have been taken to classify psoriasis include age of onset, severity of the disease, and morphologic evaluation. The latter has yielded plaque, guttate, pustular, and erythrodermic as subtypes of psoriasis. Unlike other autoimmune diseases, histopathological examination and blood tests are generally not valuable tools in making the diagnosis of psoriasis. However, on occasion, dermatopathologic evaluation may be helpful in confirming the diagnosis of psoriasis. Thus, in most cases the diagnosis of psoriasis is dependent primarily on pattern recognition that is morphologic evaluation of skin lesions and joints.     

人工智能技术及其在医学诊断中的应用及发展

本文对人工智能技术在常规医学及其医学诊断专家系统中的发展情况作了回顾，并对人工神经网络在医学诊断系统中的应用作了概述，阐明了在医学诊断系统中，主要的困难在于多种疾病的共存现象，即许多病人有着潜伏在自身内部的其它相关性疾病，而制约医学影像专家系统发展的主要原因是高级视觉系统本身的缺陷，即从医学扫描器上获得的图像数据可能是噪声和模糊的。从而增加了专家系统的复杂性，最后对人工智能技术在医学影像诊断系统中的发展前景作了展望。     

Feature selection with limited datasets.

Computer-aided diagnosis has the potential of increasing diagnostic accuracy by providing a second reading to radiologists. In many computerized schemes, numerous features can be extracted to describe suspect image regions. A subset of these features is then employed in a data classifier to determine whether the suspect region is abnormal or normal. Different subsets of features will, in general, result in different classification performances. A feature selection method is often used to determine an "optimal" subset of features to use with a particular classifier. A classifier performance measure (such as the area under the receiver operating characteristic curve) must be incorporated into this feature selection process. With limited datasets, however, there is a distribution in the classifier performance measure for a given classifier and subset of features. In this paper, we investigate the variation in the selected subset of "optimal" features as compared with the true optimal subset of features caused by this distribution of classifier performance. We consider examples in which the probability that the optimal subset of features is selected can be analytically computed. We show the dependence of this probability on the dataset sample size, the total number of features from which to select, the number of features selected, and the performance of the true optimal subset. Once a subset of features has been selected, the parameters of the data classifier must be determined. We show that, with limited datasets and/or a large number of features from which to choose, bias is introduced if the classifier parameters are determined using the same data that were employed to select the "optimal" subset of features.     

结合卷积神经网络与图卷积网络的乳腺癌病理图像分类研究

目的乳腺癌的精确诊断对于后续治疗具有重要临床意义,组织病理学分析是肿瘤诊断的金标准。卷积神经网络(convolution neural network,CNN)具有良好的局部特征提取能力,但无法有效捕捉细胞组织间的空间关系。为了有效利用这种空间关系,本文提出一种新的结合CNN与图卷积网络(graph convolution network,GCN)的病理图像分类框架,应用于乳腺癌病理图像的辅助诊断。方法首先对病理图像进行卷积及下采样得到一组特征图,然后将特征图上每个像素位置的特征向量表示为1个节点,构建具有空间结构的图,并通过GCN学习图中蕴含的空间结构特征。最后,将基于GCN的空间结构特征与基于CNN的全局特征融合,并同时对整个网络进行优化,实现基于融合特征的病理图像分类。结果本研究在提出框架下进行了3种GCN的比较,其中CNN-sGCN-fusion算法在2015生物成像挑战赛乳腺组织学数据集上获得93.53%±1.80%的准确率,在Databiox乳腺数据集上获得78.47%±5.33%的准确率。结论与传统基于CNN的病理图像分类算法相比,本文提出的结合CNN与GCN的算法有效融合了病理图像的全局特征与空间结构特征,从而提升了分类性能,具有潜在的应用可行性。     

The use of digital images in pathology

Digital images are routinely used by the publishing industry, but most diagnostic pathologists are unfamiliar with the technology and its possibilities. This review aims to explain the basic principles of digital image acquisition, storage, manipulation and use, and the possibilities provided not only in research, but also in teaching and in routine diagnostic pathology. Images of natural objects are usually expressed digitally as ‘bitmaps’—rectilinear arrays of small dots. The size of each dot can vary, but so can its information content in terms, for example, of colour, greyscale or opacity. Various file formats and compression algorithms are available. Video cameras connected to microscopes are familiar to most pathologists; video images can be converted directly to a digital form by a suitably equipped computer. Digital cameras and scanners are alternative acquisition tools of relevance to pathologists. Once acquired, a digital image can easily be subjected to the digital equivalent of any conventional darkroom manipulation and modern software allows much more flexibility, to such an extent that a new tool for scientific fraud has been created. For research, image enhancement and analysis is an increasingly powerful and affordable tool. Morphometric measurements are, after many predictions, at last beginning to be part of the toolkit of the diagnostic pathologist. In teaching, the potential to create dramatic yet informative presentations is demonstrated daily by the publishing industry; such methods are readily applicable to the classroom. The combination of digital images and the Internet raises many possibilities; for example, instead of seeking one expert diagnostic opinion, one could simultaneously seek the opinion of many, all around the globe. It is inevitable that in the coming years the use of digital images will spread from the laboratory to the medical curriculum and to the whole of diagnostic pathology. © 1997 John Wiley & Sons, Ltd.