基于改进U-Net网络的内窥镜图像烟雾净化算法

在微创手术中,电灼、激光烧蚀等操作产生的烟雾严重影响图像质量,遮挡医生视野,增加手术风险,同时也降低计算机辅助手术算法(如分割、三维重建、跟踪等)的性能,因此需要实时去除烟雾,以保持清晰的视野。提出一种基于改进U-Net网络的烟雾净化算法:为了保留更多图像细节,在U-Net网络编码器部分加入经过拉普拉斯金字塔变换的烟雾内窥镜图像;为了提升网络性能,在U-Net网络解码器部分加入CBAM注意力机制模块。以英国汉姆林(Hamlyn)中心提供的腹腔镜图像为原始数据集(训练图像15 000张,合成烟雾测试图像1 000张,真实包含雾气的测试图像129张),采用Blender软件模拟手术过程中烟雾出现的各种情况,对腹腔镜图像加入烟雾,得到合成烟雾图像,再送入模型进行训练,并进行5折交叉验证。在合成数据集上的综合测试结果如下:结构相似性指标SSIM为0.98,峰值性噪比PSNR为31.05。这两项指标说明,经过烟雾净化的图像与原图非常相似,有助于手术中还原人体内部的真实视野。模型平均运行速度为90.91 fps,在浓雾和淡雾数据集上比物理方法和以对抗神经网络为基础的各种方法效果更好。所提出的方法可以在烟雾数据集稀缺的场景为内窥镜烟雾净化算法提供高质量的解决方案,有助于医生得到清晰、开阔的手术视野。

A Desmoking Algorithm for Endoscopic Images Based on Improved U-Net

In minimally invasive surgery, the smoke generated by operations such as electrocautery and laser ablation seriously fade image quality, not only obstructs the doctor’s field of view and increases the risk of surgery, but also reduces the performance of computer-assisted surgery algorithms (such as segmentation, 3D reconstruction, tracking, etc.). Therefore, the smoke needs to be removed real time to maintain a clear vision. This paper proposed a desmoking algorithm based on the improved U-Net network. To retain more image details, we add images which undergone Laplace pyramid transformation to the encoder part; and to improve network's performance, we add attention mechanism module (CBAM) to decoder part. The laparoscopic image was provided by the Hamlyn Center and used as the original dataset (15000 training images, 1000 synthetic smoke test images, and 129 real smoke test images), and the Blender software was used to simulate various situations of smoke which was added to the laparoscopic image, and the composite image was obtained and sent to the improved U-Net model for training, and performed 5-fold cross-validation. We obtained a PSNR value of 31.05 and SSIM index of 0.98 on the composed dataset. These two indicators showed that the smoke-purified image was very similar to the original image, which helped restore the real vision of the human body during surgery. The average running time is 90.91 fps, which is applicable in a real-time medical system. The obtained results are better than the other six methods based on physical or based on GAN, therefore the proposed approach provided a high-quality solution for the endoscopic smoke removal algorithm, which helps doctors get a clear surgical field of vision.