基于实例分割和显著性计算的人工视觉多目标优化处理

通过实例分割Swin-Transformer提取分割所有前景对象,融合亮度、大小和位置图像显著性特征,提出模拟人类视觉注意机制的多特征融合注意力层级计算模型,为不同级别的前景物体采用适合的光幻视分辨率和亮度表达,实现不同的刺激编码策略进行层级优化处理。通过人工假体视觉的仿真试验表明,在所提出的多目标层级优化表达策略下,试验被试完成多目标识别的准确率、识别时间表现具有一定的显著提升。利用深度学习实例分割技术,层级化光幻视编码以仿生人类视觉选择性注意,达到增强假体植入者在复杂场景下的多物体感知,为视觉假体图像信息编码和优化处理研究的发展与应用提供参考。     

基于视觉信息处理的视觉假体研究与设计

由各种视网膜病变导致的视觉丧失已经成为影响人类生活质量最为严重的疾病之一。而随着信息科学、微电子技术、材料科学和生物医学等学科的迅猛发展与交叉,视觉假体的研究使由于视网膜色素变性(retinitis pig-mentosa)、老年性黄斑病变(age-related macular degeneration)以及其它一些眼球、视网膜、视神经和视觉皮层病变与损伤所造成失明情况下的视觉修复成为可能。随着技术的革新,视觉假体在未来发展的最大障碍已不再是假体的设计加工和手术技术,而是如何在假体与大脑之间进行有效信息传递。然而由于人类视觉系统复杂的结构,给视觉假体的制造以及视觉信息电信号映射带来困难。于是,我们将基于有限像素点模拟假体视觉信息,研究图像信息和微电极阵列刺激设计安装策略,通过对视觉假体的视觉信息处理研究,提出了一套基于生物、机械、电子一体化的视觉假体设计方案。     

视觉假体的发展与研究

目的:介绍国内外主要视觉假体研究小组的研究进展,讨论了视觉假体研究的重点与难点。方法:回顾并分析了植入在视皮层、视神经、视网膜的三种视觉假体的功能特点和其优缺点。结果:视觉假体在未来发展的最大障碍已不再是假体的设计加工和手术技术,而是如何在假体与大脑之间进行有效信息传递。结论:目前视觉假体仍未能满足失明患者日常需要,开发具有完整功能的视觉假体有着广阔的空间。     

仿真假体视觉下的人脸识别研究进展

目的：综述基于仿真假体视觉的人脸识别研究的主要进展。方法：回顾了近年来各研究小组基于仿真假体视觉的人脸识别研究进展,讨论假体视觉下人脸识别的可能性,分析了假体视觉下人脸识别的影响因素。结果：在仿真假体视觉下,分辨率和缺失率是影响人脸识别的主要因素,分辨率增加或是缺失率减小,对人脸的识别率有显著提高;其它光幻视参数中,一定程度内的灰度提高对假体下的人脸识别率有显著提高,而点大小、点间隙、对比度对其影响稍小,正方形的光幻视阵列排布的识别率高于六边形和极坐标排布。另外,图像处理策略如基于感兴趣区域的放大和对比度增强等,能帮助被试进行人脸识别。结论：由心理物理学方法得出,在假体视觉下,仍能完成基本的人脸识别任务;改变光幻视的参数或图像处理策略,会影响识别率。这些结果将帮助研究者们优化视觉假体中的信息处理和图像处理策略,并为术后康复训练提供了实验理论依据。     

视网膜假体专用集成电路研究进展

近些年来,人工视网膜假体的研究成为视觉修复领域的一个热点。文中根据不同的视网膜假体实现方案,概括介绍了视网膜假体专用集成电路的主要种类及实现方法;分析比较了视网膜上假体和视网膜下假体两种实现方式的优缺点。重点论述了视网膜假体专用集成电路中能量和数据收发、全局数字控制器以及神经驱动阵列等模块的基本原理及研究进展。最后讨论了视网膜假体专用集成电路设计上所面临的一些挑战及关注问题的展望。     

视网膜建模的研究进展

视网膜模型的建立是视网膜视觉假体中一个重要组成部分.视网膜模型是从已知的生理知识和实验数据出发模拟视网膜处理信息的功能,研究输入图像和输出神经冲动的关系.我们可以根据视网膜模型的原理将其分为硬件实现模型和算法模型.本文就对视网膜建模的国内外研究现状进行分析和讨论.     

仿真假体视觉下的运动感知研究

为了研究假体佩戴者对于目标物体的运动感知,利用Unity搭建虚拟仿真假体视觉场景,利用遮挡范式下对碰撞时间的估计,探究不同运动方向、运动速度、运动方式、目标物体形状、环境对比度以及各因素下的性别差异对仿真假体视觉下的运动感知的影响。试验结果表明,被试者对于水平方向上的估计要比垂直方向上的估计更准确;对于快速运动的估计准确度要优于相对慢速运动物体的准确度;运动方式因素对估计影响无显著性差异;相同形状下,物体越大,被试者估计的准确性越高且对多角图形的运动比圆润图形的运动更加敏感;不同环境对比度对估计也具有显著性差异;性别对于估计无显著性影响,但不同因素下的结果有区别。     

基于电刺激的外层型人工视网膜的研究进展

外层型视网膜假体采用了MEMS技术,通过植入到视网膜相应部位的电极来刺激神经节细胞,并且能够在大脑皮层视觉区域引起对应的特征电位反应,最终部分恢复生物体的视觉.这种外层型视网膜植入装置可分为眼外和眼内部分.后者功能相对重要,设计也较为复杂.它是由包含MPDA和微电极的刺激芯片及附属装置组成.本篇文章主体包括四部分:首先是视网膜假体的概况;其次是视网膜生理基础和视网膜假体理论的简介;在第三部分,为设计理念和MPDA的制造过程;最后,是对难题的讨论和未来发展的展望.     

视觉假体神经刺激器的发展现状和挑战

老年黄斑变性(AMD)和视网膜色素变性(RP)是不可治愈性视网膜疾病致盲的主要原因。针对这两种致盲疾病,研发视觉假体使盲人重见光明已经成为当前科学领域的研究热点之一。而视觉假体刺激器作为视觉假体的关键模块,对传导人工视觉编码信息、获取有效的视觉诱发具有重要的作用。本文首先介绍了多个国际知名视觉假体神经刺激器的设计方案,分析了它们的优缺点。在此基础上,我们着重论述了视觉假体神经刺激器所面临的挑战及未来的发展趋势,包括视觉假体神经刺激器的灵活性设计、低功耗设计和无线数据和能量传输设计。     

一种应用于视皮质假体图像压缩及编码系统的设计

背景:目前,视皮质假体已成为视觉修复实现的主要方法之一,图像压缩在视皮质假体的前期图像处理过程中占有重要地位。目的:探讨从空域及亮度信息两个方面分别对图像信息进行压缩的方法。方法:选取哈尔小波基对原始图像进行空间分辨率的压缩,并采用多尺度小波变换的方法对空间频率信号进行合理选取,实现信息的进一步压缩。同时,结合视觉皮质放大的特性,通过模拟视网膜对图像的前期处理,建立了非均匀压缩模型,使得图像压缩过程更接近视觉处理过程。还对亮度信息进行压缩及编码,实现了图像信息最终的压缩编码。结果与结论:构建了基于数字信号处理器的前期图像压缩及编码系统,最终实现了在10*10的点阵图阵列中进行图像信息的表达。     

Artificial vision support system (AVS2) for improved prosthetic vision

State-of-the-art and upcoming camera-driven, implanted artificial vision systems provide only tens to hundreds of electrodes, affording only limited visual perception for blind subjects. Therefore, real time image processing is crucial to enhance and optimize this limited perception. Since tens or hundreds of pixels/electrodes allow only for a very crude approximation of the typically megapixel optical resolution of the external camera image feed, the preservation and enhancement of contrast differences and transitions, such as edges, are especially important compared to picture details such as object texture. An Artificial Vision Support System (AVS²) is devised that displays the captured video stream in a pixelation conforming to the dimension of the epi-retinal implant electrode array. AVS², using efficient image processing modules, modifies the captured video stream in real time, enhancing ‘present but hidden’ objects to overcome inadequacies or extremes in the camera imagery. As a result, visual prosthesis carriers may now be able to discern such objects in their ‘field-of-view’, thus enabling mobility in environments that would otherwise be too hazardous to navigate. The image processing modules can be engaged repeatedly in a user-defined order, which is a unique capability. AVS² is directly applicable to any artificial vision system that is based on an imaging modality (video, infrared, sound, ultrasound, microwave, radar, etc.) as the first step in the stimulation/processing cascade, such as: retinal implants (i.e. epi-retinal, sub-retinal, suprachoroidal), optic nerve implants, cortical implants, electric tongue stimulators, or tactile stimulators.     

Contribution to the theory of prosthetic vision

By way of extracellular, electrical stimulation of the visual pathway, the various approaches to vision prosthesis aim to provide crude, patterned vision to individuals with profound blindness. Common to all approaches is the implantable electrode array and the rendering of phosphenes-the actuated percepts occupying the visual field of the implantee. Thus prosthetic vision may be simulated, and underlying theories as to how to render it efficacious developed. We review the field of simulated prosthetic vision. Furthermore, with retinal prosthesis in mind, we suggest a revised approach-an approach with regard to sampling theory, the vertebrate central visual pathway, and eye movements. The parallel development of prosthetic vision theory, explored via simulation and bioengineering issues surrounding neurostimulator design and implantation has bearing on the success of clinical trials by numerous groups in coming years.     

Image processing strategies based on saliency segmentation for object recognition under simulated prosthetic vision

Background and objectiveCurrent retinal prostheses can only generate low-resolution visual percepts constituted of limited phosphenes which are elicited by an electrode array and with uncontrollable color and restricted grayscale. Under this visual perception, prosthetic recipients can just complete some simple visual tasks, but more complex tasks like face identification/object recognition are extremely difficult. Therefore, it is necessary to investigate and apply image processing strategies for optimizing the visual perception of the recipients. This study focuses on recognition of the object of interest employing simulated prosthetic vision.MethodWe used a saliency segmentation method based on a biologically plausible graph-based visual saliency model and a grabCut-based self-adaptive-iterative optimization framework to automatically extract foreground objects. Based on this, two image processing strategies, Addition of Separate Pixelization and Background Pixel Shrink, were further utilized to enhance the extracted foreground objects.Resultsi) The results showed by verification of psychophysical experiments that under simulated prosthetic vision, both strategies had marked advantages over Direct Pixelization in terms of recognition accuracy and efficiency. ii) We also found that recognition performance under two strategies was tied to the segmentation results and was affected positively by the paired-interrelated objects in the scene.ConclusionThe use of the saliency segmentation method and image processing strategies can automatically extract and enhance foreground objects, and significantly improve object recognition performance towards recipients implanted a high-density implant.     

视觉假体系统组成及关键技术

如何部分或完全恢复视觉功能是目前神经工程和组织工程领域研究的热门课题,采用神经接口和神经假体方案来部分恢复视觉功能是目前主要的手段.研究者们已经对视觉形成的生理机制有着较为深入的了解,且视觉电刺激假体的出现已经在恢复功能视觉的道路上迈出了重要的一步,但假体系统的设计仍然不成熟,制约着假体的临床应用.假体系统的设计受到研究者们的广泛重视.从假体的系统组成(图像获取和处理装置、能量供给和数据传输装置、刺激电极)和假体的关键技术(微电子技术、封装)两大方面介绍了其最新的研究进展.     

Estimation of temporal resolution of object identification in human vision

The purpose of the study was to estimate the temporal processing capacity of human object identification under different stimulus conditions. Objects, either facial images or characters, were shown in a rapid sequence on a computer display using a rapid serial visual presentation (RSVP) method. One of the images was a target and the other images were distracters. The task of the observer was to identify the target. A staircase algorithm was used to determine the threshold frequency of image presentation in the RSVP sequence. The threshold frequency was determined as a function of image contrast, size, and mean luminance. The results showed that the threshold frequency, around 10 Hz for faces (100 ms per face) and about 25 Hz for characters (40 ms per character), was independent of contrast and size at medium and high contrast values, medium and large sizes, and high luminances, but decreased at very low contrasts or small sizes and medium or low levels of luminance. Computer simulations with a model, in which temporal integration limited perceptual speed, suggest that the experimentally found difference in processing time for faces and characters is not due to the physical differences of these stimulus types, but it seems that face-specific sites in the brain process facial information slower than object-specific areas process character information. Contrast, size, and luminance affect the signal-to-noise ratio and the temporal characteristics of low-level neural signal representation. Thus, the results suggest that at low contrasts, low luminances and small sizes, the processing speed of object identification is limited by low-level factors, while at high contrasts and luminances, and at large sizes, processing speed is limited by high-order processing stages. Processing speed seems to depend on stimulus type so that for faces processing is slower than for characters.     

Review of visual prosthesis (I)--retinal prosthesis

Research on how to totally or partially restore the vision has attracted more attention in the fields of neural engineering and tissue engineering. Neural interface and visual prosthesis offer alternative ways for partially repairing the visual impairment. The most widely used visual prosthetics are based on retinal stimulation. This article is a state-of-art review of the principles, technical details and the limitations of retinal prosthesis.