基于超广角荧光素眼底血管造影图像行糖尿病视网膜病变分期的多模态深度学习模型研究

目的:应用多模态深度学习模型对糖尿病视网膜病变（DR）超广角荧光素眼底血管造影（UWFA）图像进行病变程度的自动分级。方法:回顾性研究。2015年至2020年于武汉大学人民医院眼科中心就诊并接受UWFA检查的DR患者297例399只眼的798张图像作为模型的训练集和测试集。其中，无视网膜病变、非增生型DR （NPDR）...

Multimodal deep learning model for staging diabetic retinopathy based on ultra-widefield fluorescence angiography

Objective To apply the multi-modal deep learning model to automatically classify the ultra-widefield fluorescein angiography(UWFA)images of diabetic retinopathy(DR).Methods A retrospective study.From 2015 to 2020,798 images of 297 DR patients with 399 eyes who were admitted to Eye Center of Renmin Hospital of Wuhan University and were examined by UWFA were used as the training set and test set of the model.Among them,119,171,and 109 eyes had no retinopathy,non-proliferative DR(NPDR),and proliferative DR(PDR),respectively.Localization and assessment of fluorescein leakage and non-perfusion regions in early and late orthotopic images of UWFA in DR-affected eyes by jointly optimizing CycleGAN and a convolutional neural network(CNN)classifier,an image-level supervised deep learning model.The abnormal images with lesions were converted into normal images with lesions removed using the improved CycleGAN,and the difference images containing the lesion areas were obtained;the difference images were classified by the CNN classifier to obtain the prediction results.A five-fold cross-test was used to evaluate the classification accuracy of the model.Quantitative analysis of the marker area displayed by the differential images was performed to observe the correlation between the ischemia index and leakage index and the severity of DR.Results The generated fake normal image basically removed all the lesion areas while retaining the normal vascular structure;the difference images intuitively revealed the distribution of biomarkers;the heat icon showed the leakage area,and the location was basically the same as the lesion area in the original image.The results of the five-fold cross-check showed that the average classification accuracy of the model was 0.983.Further quantitative analysis of the marker area showed that the ischemia index and leakage index were significantly positively correlated with the severity of DR(β=6.088,10.850;P<0.001).Conclusion The constructed multimodal joint optimization model can accurately classify NPDR and PDR and precisely locate potential biomarkers.