Estimation of simulated phosphene size based on tactile perception

Clinical trials have successfully shown that a visual prosthesis can elicit visual perception (phosphenes) in the visual field. Psychophysical studies based on simulated prosthetic vision offer an effective means to evaluate and refine prosthetic vision. We designed three experiments to examine the effect of phosphene luminance, flicker rate, and eccentricity on the ability to estimate simulated phosphene sizes using tactile perception. Thirty subjects participated in the three experiments. There was a linear increase in reported size as visual stimulus size increased. Judgment was significantly affected by stimulus luminance and eccentricity (P < 0.05) but not by flicker rates. Brighter stimuli were perceived as being larger, and the more eccentric the position, the larger the estimated size. These simulation studies, although idealized, suggested that tactile perception is a potential way to estimate phosphene sizes.     

Dispersion and Accuracy of Simulated Phosphene Positioning Using Tactile Board

The evaluation of phosphene is one of the most important things after the electrode array's implantation. Some experiments have been used to study the basic rules on people using their tactile perception in describing the characteristics of simulated phosphenes, such as the dispersion, accuracy, and response time. In order to describe the simulated phosphenes more systematically and scientifically, an improved phosphene‐positioning method using tactile board was designed in this study. Using tactile board to guide the tactile perception, the dispersion was limited to less than 6 mm, while the largest mean error was 8.1 mm, which was nearly equal to the minimal resolution of tactile board (8 mm). The response time greatly increased because of the standard processes in using tactile board. Moreover, the long‐term experiment to repeat the procedure could improve and sustain the subjects' performance in good results.     

Phosphene Object Perception Employs Holistic Processing During Early Visual Processing Stage

Psychophysical studies have verified the possibility of recovering the visual ability by the form of low‐resolution format of images, that is, phosphene‐based representations. Our previous study has found that early visual processing for phosphene patterns is configuration based. This study further investigated the configural processing mechanisms of prosthetic vision by analyzing the event‐related potential components (P1 and N170) in response to phosphene face and non‐face stimuli. The results reveal that the coarse processing of phosphenes involves phosphene‐specific holistic processing that recovers separated phosphenes into a gestalt; low‐level feature processing of phosphenes is also enhanced compared with that of normal stimuli due to increased contrast borders introduced by phosphenes; while fine processing of phosphene stimuli is impaired reflected by reduced N170 amplitude because of the degraded detailed features in the low‐resolution format representations. Therefore, we suggest that strategies that can facilitate the specific holistic processing stages of prosthetic vision should be considered in order to improve the performance when designing the visual prosthesis system.     

Configuration‐Based Processing of Phosphene Pattern Recognition for Simulated Prosthetic Vision

Visual prosthesis can elicit phosphenes by stimulating the retina, optic nerve, or visual cortex along the visual pathway. Psychophysical studies have demonstrated that visual function can be partly recovered with phosphene‐based prosthetic vision. This study investigated the cognitive process of prosthetic vision through a face recognition task. Both behavioral response and the face‐specific N170 component of event‐related potential were analyzed in the presence of face and non‐face stimuli with natural and simulated prosthetic vision. Our results showed that: (i) the accuracy of phosphene face recognition was comparable with that of the normal one when phosphene grid increased to 25 × 21 or more; (ii) shorter response time was needed for phosphene face recognition; and (iii) the N170 component was delayed and enhanced under phosphene stimuli. It was suggested that recognition of phosphene patterns employ a configuration‐based holistic processing mechanism with a distinct substage unspecific to faces.     

Object Recognition Under Distorted Prosthetic Vision

Psychophysical studies have reported the efficacy of phosphene‐based prosthetic vision in partly recovering the visual function of blind individuals. However, results by far have been based on evenly aligned phosphene arrays, which neglected the complicated visuotopy in the visual prosthesis system. In this study, we investigated how the objects were recognized under the stimuli with distorted phosphene arrays simulated by transformations of barrel distortion, rotation, or translation. The results revealed that distortions significantly decreased the accuracy of categorization (CA) and showed distinct interactive effects with the factors of object category and phosphene array density. Moreover, the CA changed differently with the increase of distortion levels. Regression analysis suggested a phosphene array of at least 10 × 10 be the essential for achieving a CA over the threshold value (CA_t = 62.5%) under distorted prosthetic vision. It is recommended that discriminative features be extracted to improve the performance of prosthetic vision.     

Phosphenes elicited by subcortical stimulation in man]

Electrical stimulation of a point in the visual pathway can evoke a visual sensation which is called a phosphene. The phosphenes elicited by intracerebral stimulation were investigated in twenty-three subjects. One hundred and seven phosphenes were reported and all of them appeared in the visual field contralateral to the side of stimulation. The exception was a single case where a diffuse flashing sensation appeared in the whole visual field. Thirteen patients reported white phosphenes and nine patients reported coloured phosphenes. In the medial area (10-15 mm from the midline) of the occipital lobe, stimuli above the calcarine fissure resulted in phosphenes in the lower quadrant of the visual field. In the lateral area (16-32 mm from the midline), however, stimuli above the level of the calcarine fissure tended to produce phosphenes in the upper quadrant. These findings appear to conflict with traditional concept of the physiological anatomy of the visual pathway in man. The possible mechanism of this phenomenon produced by intracerebral stimulation is discussed in relation to the phosphenes produced by cortical stimulation.     

Enhanced excitability of the human visual cortex induced by short-term light deprivation

Long-term deprivation of visual input for several days or weeks leads to marked changes in the excitability and function of the occipital cortex. The time course of these changes is poorly understood. In this study, we addressed the question whether a short period of light deprivation (minutes to a few hours) can elicit such changes in humans. Noninvasive transcranial magnetic stimulation (TMS) of the human occipital cortex can evoke the perception of flashes or spots of light (phosphenes). To assess changes in visual cortex excitability following light deprivation, we measured the minimum intensity of stimulation required to elicit phosphenes (phosphene threshold) and the number of phosphenes elicited by different TMS stimulus intensities (stimulus-response curves). A reduced phosphene threshold was detected 45 min after the onset of light deprivation and persisted for the entire deprivation period (180 min). Following re-exposure to light, phosphene thresholds returned to predeprivation values over 120 min. Stimulus-response curves were significantly enhanced in association with this intervention. In a second experiment, we studied the effects of light deprivation on functional magnetic resonance imaging (fMRI) signals elicited by photic stimulation. fMRI results showed increased visual cortex activation after 60 min of light deprivation that persisted following 30 min of re-exposure to light. Our results demonstrated a substantial increase in visual cortex excitability. These changes may underlie behavioral gains reported in humans and animals associated with light deprivation.     

Recognition of Similar Objects Using Simulated Prosthetic Vision

Due to the limitations of existing techniques, even the most advanced visual prostheses, using several hundred electrodes to transmit signals to the visual pathway, restrict sensory function and visual information. To identify the bottlenecks and guide prosthesis designing, psychophysics simulations of a visual prosthesis in normally sighted individuals are desirable. In this study, psychophysical experiments of discriminating objects with similar profiles were used to test the effects of phosphene array parameters (spatial resolution, gray scale, distortion, and dropout rate) on visual information using simulated prosthetic vision. The results showed that the increase in spatial resolution and number of gray levels and the decrease in phosphene distortion and dropout rate improved recognition performance, and the accuracy is 78.5% under the optimum condition (resolution: 32 × 32, gray level: 8, distortion: k = 0, dropout: 0%). In combined parameter tests, significant facial recognition accuracy was achieved for all the images with k = 0.1 distortion and 10% dropout. Compared with other experiments, we find that different objects do not show specific sensitivity to the changes of parameters and visual information is not nearly enough even under the optimum condition. The results suggests that higher spatial resolution and more gray levels are required for visual prosthetic devices and further research on image processing strategies to improve prosthetic vision is necessary, especially when the wearers have to accomplish more than simple visual tasks.     

Estimation of Phosphene Spatial Variability for Visual Prosthesis Applications

Visual prostheses are the focus of intensive research efforts to restore some measure of useful vision to blind or near‐blind patients. The development of such technology is being guided to an extent by tools that simulate prosthesis behavior for healthy sighted subjects in order to assess system requirements and configurations. These simulators, however, typically assume purely deterministic phosphene properties and thus do not apply any variability to phosphene size, intensity, or location. We address this issue by presenting data on phosphene variability measured in a blind human subject fitted with an optic nerve prosthesis. In order to correct for normal limitations in human‐pointing accuracy, the experimental conditions were repeated with sighted subjects. We conclude that identical optic nerve stimulations can result in phosphenes whose perceived locations vary by up to 5° of deviation angle and 10° of position angle. The consistency of phosphenes presented in the peripheral field of view can vary by an additional 3°.     

A model for intracortical visual prosthesis research

In the field of visual prosthesis research, it has generally been held that animal models are limited to testing the safety of implantable hardware due to the inability of the animal to provide a linguistic report of perceptions. In contrast, vision scientists make extensive use of trained animal models to investigate the links between visual stimuli, neural activities, and perception. We describe an animal model for cortical visual prosthesis research in which novel animal psychophysical testing has been employed to compensate for the lack of a linguistic report. One hundred and fifty-two intracortical microelectrodes were chronically implanted in area V1 of a male macaque. Receptive field mapping was combined with eye-tracking to develop a reward-based training procedure. The animal was trained to use electrically induced point-flash percepts, called phosphenes, in performing a memory saccade task. It is our long-term goal to use this animal model to investigate stimulation strategies in developing a multichannel sensory cortical interface.     

Image processing strategies dedicated to visual cortical stimulators: a survey

Multi-electrode devices are constantly evolving toward a state where complexity and reliability are adequate for providing a breakthrough in visual cortical stimulation allowing the blind to recover partial vision. Yet few research teams have focused on the development of the front-end subsystem that transforms an input image from a camera into stimulation commands. This article collects state-of-the-art knowledge about the appearance and organization of phosphenes, and previous work in image processing dedicated to visual cortical stimulation. Observations and hypothesis about important issues are highlighted, and six image processing strategies that could be used in such a subsystem are presented, from the most optimistic that use brightness modulation to emulate grayscale to the most conservative that use only on/off phosphene evocation.     

The perceptual and functional consequences of parietal top-down modulation on the visual cortex

The posterior parietal cortex (PPC) has been proposed to play a critical role in exerting top-down influences on occipital visual areas. By inducing activity in the PPC (angular gyrus) using transcranial magnetic stimulation (TMS), and using the phosphene threshold as a measure of visual cortical excitability, we investigated the functional role of this region in modulating the activity of the visual cortex. When triple-pulses of TMS were applied over the PPC unilaterally, the intensity of stimulation required to elicit a phosphene from the visual cortex (area V1/V2) was reduced, indicating an increase in visual cortical excitability. The increased excitability that was observed with unilateral TMS was abolished when TMS was applied over the PPC bilaterally. Our results provide a demonstration of the top-down modulation exerted by the PPC on the visual cortex and show that these effects are subject to interhemispheric competition.     

Adaptation to Phosphene Parameters Based on Multi‐Object Recognition Using Simulated Prosthetic Vision

Retinal prostheses for the restoration of functional vision are under development and visual prostheses targeting proximal stages of the visual pathway are also being explored. To investigate the experience with visual prostheses, psychophysical experiments using simulated prosthetic vision in normally sighted individuals are necessary. In this study, a helmet display with real‐time images from a camera attached to the helmet provided the simulated vision, and experiments of recognition and discriminating multiple objects were used to evaluate visual performance under different parameters (gray scale, distortion, and dropout). The process of fitting and training with visual prostheses was simulated and estimated by adaptation to the parameters with time. The results showed that the increase in the number of gray scale and the decrease in phosphene distortion and dropout rate improved recognition performance significantly, and the recognition accuracy was 61.8 ± 7.6% under the optimum condition (gray scale: 8, distortion: k = 0, dropout: 0%). The adaption experiments indicated that the recognition performance was improved with time and the effect of adaptation to distortion was greater than dropout, which implies the difference of adaptation mechanism to the two parameters.     

Retinotopy with coordinates of lateral occipital cortex in humans

OBJECT: The lateral occipital cortex in humans is known as the "extrastriate visual cortex." It is, however, an unexplored field of research, and the anatomical nomenclature for its surface has still not been standardized. This study was designed to investigate whether the lateral occipital cortex in humans has retinotopic representation. METHODS: Four right-handed patients with a diagnosis of intractable epilepsy from space-occupying lesions in the occipital lobe or epilepsy originating in the occipital lobe received permanently implanted subdural electrodes. Electrical cortical stimulation was applied directly applied to the brain through metal electrodes by using a biphasic stimulator. The location of each electrode was measured on a lateral skull x-ray study. Each patient considered a whiteboard with vertical and horizontal median lines. The patient was asked to look at the midpoint on the whiteboard. If a visual hallucination or illusion occurred, the patient recorded its outline, shape, color, location, and motion on white paper one tenth the size of, and with vertical and horizontal median lines similar to those on, the whiteboard. Polar angles and eccentricities of the midpoints of the phosphenes from the coordinate origin were measured on the paper. On stimulation of the lateral occipital lobe, 44 phosphenes occurred. All phosphenes were circular or dotted, with a diameter of approximately 1 cm, except one that was like a curtain in the peripheral end of the upper and lower visual fields on stimulation of the parietooccipital region. All phosphenes appeared in the visual field contralateral to the cerebral hemisphere stimulated. On stimulation of the lateral occipital lobe, 22 phosphenes moved centrifugally or toward a horizontal line. From three-dimensional scatterplots and contour maps of the polar angles and eccentricities in relation to the x-ray coordinates of the electrodes, one can infer that the lateral occipital cortex in humans has retinotopic representation. CONCLUSIONS: The authors found that phosphenes induced by electrical cortical stimulation of the lateral occipital cortex represent retinotopy. From these results one can assert that visual field representation with retinotopic relation exists in the extrastriate visual cortex.     

Human tactile perception as a standard for artificial tactile sensing--a review

In this paper, we examine the most important features of human skin tactile properties with special emphasis on the characteristics which are vital in the design of artificial systems. Contrary to the visual and auditory senses, the touch signal is not a well-defined quantity. As a result, the researchers of this field are still dealing with the basics of collecting the most relevant data. Following this, mimicking the sense of touch by producing artificial tactile skin is a challenging process. Although the sense of touch is widely distributed all over the human body, the tactile perception in the human hand is of great importance in terms of surgical and medical robotics applications. In this study, the role of various mechanoreceptors in the human hand, such as, RA, SA I, SA II, and PC units are discussed in relation to the stimuli like force, position, softness, and surface texture. Taking human hand as a suitable tactile model, the necessary engineering features of an artificial tactile sensor, such as, spatial and temporal resolutions, force sensitivity, and linearity, are being reviewed. In this work, we also report on the current and possible future applications of tactile sensors in various surgical procedures.     

Pattern recognition with the optic nerve visual prosthesis

A volunteer with retinitis pigmentosa and no residual vision was chronically implanted with an optic nerve electrode connected to an implanted neurostimulator and antenna. An external controller with telemetry was used for electrical activation of the nerve which resulted in phosphene perception. Open-loop stimulation allowed the collection of phosphene attributes and the ability to elicit perception of simple geometrical patterns. Low perception thresholds allowed for large current intensity range within safety limits. In a closed-loop paradigm, the volunteer was using a head-worn video camera to explore a projection screen. The volunteer underwent performance evaluation during the course of a training program with 45 simple patterns. After learning, the volunteer reached a recognition score of 63% with a processing time of 60 s. Mean performance in orientation discrimination reached 100% with a processing time of 8 s.     

The effect of visual experience on the development of functional architecture in hMT+

We investigated whether the visual hMT+ cortex plays a role in supramodal representation of sensory flow, not mediated by visual mental imagery. We used functional magnetic resonance imaging to measure neural activity in sighted and congenitally blind individuals during passive perception of optic and tactile flows. Visual motion-responsive cortex, including hMT+, was identified in the lateral occipital and inferior temporal cortices of the sighted subjects by response to optic flow. Tactile flow perception in sighted subjects activated the more anterior part of these cortical regions but deactivated the more posterior part. By contrast, perception of tactile flow in blind subjects activated the full extent, including the more posterior part. These results demonstrate that activation of hMT+ and surrounding cortex by tactile flow is not mediated by visual mental imagery and that the functional organization of hMT+ can develop to subserve tactile flow perception in the absence of any visual experience. Moreover, visual experience leads to a segregation of the motion-responsive occipitotemporal cortex into an anterior subregion involved in the representation of both optic and tactile flows and a posterior subregion that processes optic flow only.     

Simplification of Visual Rendering in Simulated Prosthetic Vision Facilitates Navigation

Visual neuroprostheses are still limited and simulated prosthetic vision (SPV) is used to evaluate potential and forthcoming functionality of these implants. SPV has been used to evaluate the minimum requirement on visual neuroprosthetic characteristics to restore various functions such as reading, objects and face recognition, object grasping, etc. Some of these studies focused on obstacle avoidance but only a few investigated orientation or navigation abilities with prosthetic vision. The resolution of current arrays of electrodes is not sufficient to allow navigation tasks without additional processing of the visual input. In this study, we simulated a low resolution array (15 × 18 electrodes, similar to a forthcoming generation of arrays) and evaluated the navigation abilities restored when visual information was processed with various computer vision algorithms to enhance the visual rendering. Three main visual rendering strategies were compared to a control rendering in a wayfinding task within an unknown environment. The control rendering corresponded to a resizing of the original image onto the electrode array size, according to the average brightness of the pixels. In the first rendering strategy, vision distance was limited to 3, 6, or 9 m, respectively. In the second strategy, the rendering was not based on the brightness of the image pixels, but on the distance between the user and the elements in the field of view. In the last rendering strategy, only the edges of the environments were displayed, similar to a wireframe rendering. All the tested renderings, except the 3 m limitation of the viewing distance, improved navigation performance and decreased cognitive load. Interestingly, the distance‐based and wireframe renderings also improved the cognitive mapping of the unknown environment. These results show that low resolution implants are usable for wayfinding if specific computer vision algorithms are used to select and display appropriate information regarding the environment.     

The microsystems based visual prosthesis for optic nerve stimulation

The microsystems based visual prosthesis (MiViP) visual prosthesis generates visual perceptions well below safety and stimulator saturation limits. These perceptions, called phosphenes, are of reasonably small size and are broadly distributed in the visual field. They can thus be used to convey useful visual information. Psychophysical evaluations are being performed in order to assess the implantee's benefits in the use of the MiViP optic nerve visual prosthesis. In a pattern-recognition task, the performance improved regularly with practice with an increasing score and a decreasing delay to recognition. These observations open the way toward an evaluation of general mobility improvement with the portable system. In conclusion, the results obtained so far still support the potential usefulness of the optic nerve visual prosthesis. A low-resolution artificial vision can be expected from the prosthesis after extensive training.     

Strabismic suppression is mediated by inhibitory interactions in the primary visual cortex

Most strabismic observers do not suffer from double vision because of suppression from conscious perception of 1 of the 2 eyes' conflicting views. Direct evidence for the site and neural substrate of strabismic suppression has not been available so far, although psychophysical data suggest a cortical origin. On the other hand, cross-orientation suppression among conflicting stimuli presented monocularly has recently been shown to have a strong thalamic component. Here we present evidence, using both visual stimulation and pharmacological techniques, that strabismic suppression occurs in the primary visual cortex and involves gamma-amino butyric acid (GABA)-mediated inhibition. We show that its dependency on the drift rate of the suppressing stimulus is consistent with a cortical origin; unlike monocular cross-orientation suppression, it cannot be evoked by very fast-moving stimuli. Furthermore, strabismic suppression is greatly reduced when GABAergic inhibition is locally blocked by the GABA(A) antagonist bicuculline.