A survey on active learning and human-in-the-loop deep learning for medical image analysis

Fully automatic deep learning has become the state-of-the-art technique for many tasks including image acquisition, analysis and interpretation, and for the extraction of clinically useful information for computer-aided detection, diagnosis, treatment planning, intervention and therapy. However, the unique challenges posed by medical image analysis suggest that retaining a human end-user in any deep learning enabled system will be beneficial. In this review we investigate the role that humans might play in the development and deployment of deep learning enabled diagnostic applications and focus on techniques that will retain a significant input from a human end user. Human-in-the-Loop computing is an area that we see as increasingly important in future research due to the safety-critical nature of working in the medical domain. We evaluate four key areas that we consider vital for deep learning in the clinical practice: (1) Active Learning to choose the best data to annotate for optimal model performance; (2) Interaction with model outputs - using iterative feedback to steer models to optima for a given prediction and offering meaningful ways to interpret and respond to predictions; (3) Practical considerations - developing full scale applications and the key considerations that need to be made before deployment; (4) Future Prospective and Unanswered Questions - knowledge gaps and related research fields that will benefit human-in-the-loop computing as they evolve. We offer our opinions on the most promising directions of research and how various aspects of each area might be unified towards common goals.     

Cross-dimensional transfer learning in medical image segmentation with deep learning

Over the last decade, convolutional neural networks have emerged and advanced the state-of-the-art in various image analysis and computer vision applications. The performance of 2D image classification networks is constantly improving and being trained on databases made of millions of natural images. Conversely, in the field of medical image analysis, the progress is also remarkable but has mainly slowed down due to the relative lack of annotated data and besides, the inherent constraints related to the acquisition process. These limitations are even more pronounced given the volumetry of medical imaging data. In this paper, we introduce an efficient way to transfer the efficiency of a 2D classification network trained on natural images to 2D, 3D uni- and multi-modal medical image segmentation applications. In this direction, we designed novel architectures based on two key principles: weight transfer by embedding a 2D pre-trained encoder into a higher dimensional U-Net, and dimensional transfer by expanding a 2D segmentation network into a higher dimension one. The proposed networks were tested on benchmarks comprising different modalities: MR, CT, and ultrasound images. Our 2D network ranked first on the CAMUS challenge dedicated to echo-cardiographic data segmentation and surpassed the state-of-the-art. Regarding 2D/3D MR and CT abdominal images from the CHAOS challenge, our approach largely outperformed the other 2D-based methods described in the challenge paper on Dice, RAVD, ASSD, and MSSD scores and ranked third on the online evaluation platform. Our 3D network applied to the BraTS 2022 competition also achieved promising results, reaching an average Dice score of 91.69% (91.22%) for the whole tumor, 83.23% (84.77%) for the tumor core and 81.75% (83.88%) for enhanced tumor using the approach based on weight (dimensional) transfer. Experimental and qualitative results illustrate the effectiveness of our methods for multi-dimensional medical image segmentation.     

A survey on incorporating domain knowledge into deep learning for medical image analysis

Although deep learning models like CNNs have achieved great success in medical image analysis, the small size of medical datasets remains a major bottleneck in this area. To address this problem, researchers have started looking for external information beyond current available medical datasets. Traditional approaches generally leverage the information from natural images via transfer learning. More recent works utilize the domain knowledge from medical doctors, to create networks that resemble how medical doctors are trained, mimic their diagnostic patterns, or focus on the features or areas they pay particular attention to. In this survey, we summarize the current progress on integrating medical domain knowledge into deep learning models for various tasks, such as disease diagnosis, lesion, organ and abnormality detection, lesion and organ segmentation. For each task, we systematically categorize different kinds of medical domain knowledge that have been utilized and their corresponding integrating methods. We also provide current challenges and directions for future research.     

Recent advances in medical image processing for the evaluation of chronic kidney disease

Assessment of renal function and structure accurately remains essential in the diagnosis and prognosis of Chronic Kidney Disease (CKD). Advanced imaging, including Magnetic Resonance Imaging (MRI), Ultrasound Elastography (UE), Computed Tomography (CT) and scintigraphy (PET, SPECT) offers the opportunity to non-invasively retrieve structural, functional and molecular information that could detect changes in renal tissue properties and functionality. Currently, the ability of artificial intelligence to turn conventional medical imaging into a full-automated diagnostic tool is widely investigated. In addition to the qualitative analysis performed on renal medical imaging, texture analysis was integrated with machine learning techniques as a quantification of renal tissue heterogeneity, providing a promising complementary tool in renal function decline prediction. Interestingly, deep learning holds the ability to be a novel approach of renal function diagnosis. This paper proposes a survey that covers both qualitative and quantitative analysis applied to novel medical imaging techniques to monitor the decline of renal function. First, we summarize the use of different medical imaging modalities to monitor CKD and then, we show the ability of Artificial Intelligence (AI) to guide renal function evaluation from segmentation to disease prediction, discussing how texture analysis and machine learning techniques have emerged in recent clinical researches in order to improve renal dysfunction monitoring and prediction. The paper gives a summary about the role of AI in renal segmentation.     

Recent advances in miRNA-based diagnostic applications

This report highlights presentations and discussions of the 8th Cambridge Health Institute conference on miRNA in Human Disease and Development held in Cambridge (MA, USA) on 12-13 March 2012. The areas covered included development of miRNA-based diagnostic and therapeutic applications, as well as mechanistic miRNA studies in cancer, cardiovascular disease and neurodegenerative conditions. There were also several industry-sponsored events showcasing miRNA technologies.     

Large-Scale medical image analytics: Recent methodologies,applications and Future directions

Despite the ever-increasing amount and complexity of annotated medical image data, the development of large-scale medical image analysis algorithms has not kept pace with the need for methods that bridge the semantic gap between images and diagnoses. The goal of this position paper is to discuss and explore innovative and large-scale data science techniques in medical image analytics, which will benefit clinical decision-making and facilitate efficient medical data management. Particularly, we advocate that the scale of image retrieval systems should be significantly increased at which interactive systems can be effective for knowledge discovery in potentially large databases of medical images. For clinical relevance, such systems should return results in real-time, incorporate expert feedback, and be able to cope with the size, quality, and variety of the medical images and their associated metadata for a particular domain. The design, development, and testing of the such framework can significantly impact interactive mining in medical image databases that are growing rapidly in size and complexity and enable novel methods of analysis at much larger scales in an efficient, integrated fashion.     

基于深度学习与多模态医学影像融合识别阈下抑郁患者

目的 基于深度学习（DL）卷积神经网络（CNN）算法，利用医学影像数据实现识别阈下抑郁（StD）患者。方法 对56例StD患者和70名正常人采集MRI和fMRI数据，分别输入所构建的CNN，利用网络融合技术对2种不同模态数据进行综合分析，得到分类结果；最后调整网络结构与模型参数，实现分类效果最优化。结果 单独MRI数据模型分类精度为73.02%，单独fMRI数据模型分类精度为65.08%；2种模态结合，最终分类精度升至78.57%。结论 利用DL可识别StD患者与正常人；采用多种模态输入法可提高分类准确度。     

Super-resolution and segmentation deep learning for breast cancer histopathology image analysis

Traditionally, a high-performance microscope with a large numerical aperture is required to acquire high-resolution images. However, the images’ size is typically tremendous. Therefore, they are not conveniently managed and transferred across a computer network or stored in a limited computer storage system. As a result, image compression is commonly used to reduce image size resulting in poor image resolution. Here, we demonstrate custom convolution neural networks (CNNs) for both super-resolution image enhancement from low-resolution images and characterization of both cells and nuclei from hematoxylin and eosin (H&E) stained breast cancer histopathological images by using a combination of generator and discriminator networks so-called super-resolution generative adversarial network-based on aggregated residual transformation (SRGAN-ResNeXt) to facilitate cancer diagnosis in low resource settings. The results provide high enhancement in image quality where the peak signal-to-noise ratio and structural similarity of our network results are over 30 dB and 0.93, respectively. The derived performance is superior to the results obtained from both the bicubic interpolation and the well-known SRGAN deep-learning methods. In addition, another custom CNN is used to perform image segmentation from the generated high-resolution breast cancer images derived with our model with an average Intersection over Union of 0.869 and an average dice similarity coefficient of 0.893 for the H&E image segmentation results. Finally, we propose the jointly trained SRGAN-ResNeXt and Inception U-net Models, which applied the weights from the individually trained SRGAN-ResNeXt and inception U-net models as the pre-trained weights for transfer learning. The jointly trained model’s results are progressively improved and promising. We anticipate these custom CNNs can help resolve the inaccessibility of advanced microscopes or whole slide imaging (WSI) systems to acquire high-resolution images from low-performance microscopes located in remote-constraint settings.     

Recent advances in basic and clinical nanomedicine

Nanomedicine is a global business enterprise. Industry and governments clearly are beginning to envision nanomedicine's enormous potential. A clear definition of nanotechnology is an issue that requires urgent attention. This problem exists because nanotechnology represents a cluster of technologies, each of which may have different characteristics and applications. Although numerous novel nanomedicine-related applications are under development or nearing commercialization, the process of converting basic research in nanomedicine into commercially viable products will be long and difficult. Although realization of the full potential of nanomedicine may be years or decades away, recent advances in nanotechnology-related drug delivery, diagnosis, and drug development are beginning to change the landscape of medicine. Site-specific targeted drug delivery and personalized medicine are just a few concepts that are on the horizon.     

深度学习CT图像迭代重建及其用于儿童CT进展

人工智能在分割、重建医学及图像处理等方面均发挥重要作用。儿童CT检查应遵循尽可能低辐射剂量原则，即在低辐射剂量下最大限度保持或获得更高图像质量。本文对基于人工智能的深度学习CT图像迭代重建技术及其用于儿童CT进展进行综述。     

Recent advances in clinical low-vision care

In the past decade there has been a growth in the awareness of the impact that low vision has on the individual and on society in general. The clinical eye-care professions have become more conscious of their responsibility to assist partially sighted patients and low-vision care is becoming a specialty area within optometry and ophthalmology. This paper reviews recent advances in the optical devices and prescribing procedures that have contributed to improving the care that is available to assist low-vision patients with distance-vision tasks, near-vision tasks and visual-field problems.     

Guidelines and evaluation of clinical explainable AI in medical image analysis

Explainable artificial intelligence (XAI) is essential for enabling clinical users to get informed decision support from AI and comply with evidence-based medical practice. Applying XAI in clinical settings requires proper evaluation criteria to ensure the explanation technique is both technically sound and clinically useful, but specific support is lacking to achieve this goal. To bridge the research gap, we propose the Clinical XAI Guidelines that consist of five criteria a clinical XAI needs to be optimized for. The guidelines recommend choosing an explanation form based on Guideline 1 (G1) Understandability and G2 Clinical relevance. For the chosen explanation form, its specific XAI technique should be optimized for G3 Truthfulness, G4 Informative plausibility, and G5 Computational efficiency. Following the guidelines, we conducted a systematic evaluation on a novel problem of multi-modal medical image explanation with two clinical tasks, and proposed new evaluation metrics accordingly. Sixteen commonly-used heatmap XAI techniques were evaluated and found to be insufficient for clinical use due to their failure in G3 and G4. Our evaluation demonstrated the use of Clinical XAI Guidelines to support the design and evaluation of clinically viable XAI.     

Recent advances in clinical use of opioids

Opioid therapy for pain is the subject of numerous randomized clinical trials. Opioids are being developed for delivery by a wide variety of mechanisms. New opioids are becoming available for clinical use. This review surveys recent developments in these clinical trials and provides an overview of what may be expected in the near future for opioid management of pain.     

Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation

The medical imaging literature has witnessed remarkable progress in high-performing segmentation models based on convolutional neural networks. Despite the new performance highs, the recent advanced segmentation models still require large, representative, and high quality annotated datasets. However, rarely do we have a perfect training dataset, particularly in the field of medical imaging, where data and annotations are both expensive to acquire. Recently, a large body of research has studied the problem of medical image segmentation with imperfect datasets, tackling two major dataset limitations: scarce annotations where only limited annotated data is available for training, and weak annotations where the training data has only sparse annotations, noisy annotations, or image-level annotations. In this article, we provide a detailed review of the solutions above, summarizing both the technical novelties and empirical results. We further compare the benefits and requirements of the surveyed methodologies and provide our recommended solutions. We hope this survey article increases the community awareness of the techniques that are available to handle imperfect medical image segmentation datasets.     

Explainable artificial intelligence (XAI) in deep learning-based medical image analysis

With an increase in deep learning-based methods, the call for explainability of such methods grows, especially in high-stakes decision making areas such as medical image analysis. This survey presents an overview of explainable artificial intelligence (XAI) used in deep learning-based medical image analysis. A framework of XAI criteria is introduced to classify deep learning-based medical image analysis methods. Papers on XAI techniques in medical image analysis are then surveyed and categorized according to the framework and according to anatomical location. The paper concludes with an outlook of future opportunities for XAI in medical image analysis.     

Recent advances in PPARs research--basic and clinical aspects

PPARs (peroxisome proliferator-activated receptors), PPARalpha, beta/delta and gamma, play crucial roles in regulation of glucose and lipid metabolism, energy homeostasis and atherosclerosis. Specific and common actions of these three PPARs have recently been elucidated using mice models and specific, dual, or pan-agonists. A number of drugs targeted to PPARs are being developed for metabolic disorders such as diabetes and hyperlipidemia and atherosclerosis diseases. This review focuses the recent advances in PPARs research from basic and clinical aspects.     

深度学习在医学影像中的应用

医学影像是放射科医生做出医学诊断的重要依据。但随着医学影像技术的快速发展, 逐渐增多的影像图像和复杂的图像信息对医生的工作产生了巨大的挑战。而深度学习是人工智能研究中最热门的领域, 在处理大数据和提取有效信息方面具有优势, 因此逐渐成为分析医学影像方面的首选方法。本文阐述了深度学习的概念, 并简要总结深度学习在医学影像中的常见模型, 包括卷积神经网络、循环神经网络、深度置信网络和自动编码器。卷积神经网络的基本结构是卷积层、池化层和全连接层; 循环神经网络由输入层、隐藏层和输出层组成; 深度置信网络的基础是玻尔兹曼机; 自动编码器包含编码层、隐藏层和解码层。通过对CT肺结节和MRI脑部疾病的分类, 阐明目前深度学习在疾病自动分类上准确性较高; 通过分割左心室、椎旁肌肉和肝脏的结构, 可见深度学习方法在医学图像分割上与人为分割具有一致性; 深度学习在肺结节和乳腺癌疾病的检测上已相对成熟。但目前为止, 仍存在标注的样本量少和过拟合的问题, 希望通过共享图像数据库来解决此问题。总之, 深度学习在医学影像中具有广阔前景, 且对临床医生的工作具有重大意义。