Simulated Prosthetic Vision: The Benefits of Computer‐Based Object Recognition and Localization

Clinical trials with blind patients implanted with a visual neuroprosthesis showed that even the simplest tasks were difficult to perform with the limited vision restored with current implants. Simulated prosthetic vision (SPV) is a powerful tool to investigate the putative functions of the upcoming generations of visual neuroprostheses. Recent studies based on SPV showed that several generations of implants will be required before usable vision is restored. However, none of these studies relied on advanced image processing. High‐level image processing could significantly reduce the amount of information required to perform visual tasks and help restore visuomotor behaviors, even with current low‐resolution implants. In this study, we simulated a prosthetic vision device based on object localization in the scene. We evaluated the usability of this device for object recognition, localization, and reaching. We showed that a very low number of electrodes (e.g., nine) are sufficient to restore visually guided reaching movements with fair timing (10 s) and high accuracy. In addition, performance, both in terms of accuracy and speed, was comparable with 9 and 100 electrodes. Extraction of high level information (object recognition and localization) from video images could drastically enhance the usability of current visual neuroprosthesis. We suggest that this method—that is, localization of targets of interest in the scene—may restore various visuomotor behaviors. This method could prove functional on current low‐resolution implants. The main limitation resides in the reliability of the vision algorithms, which are improving rapidly.     

Recognition of Similar Objects Using Simulated Prosthetic Vision

Due to the limitations of existing techniques, even the most advanced visual prostheses, using several hundred electrodes to transmit signals to the visual pathway, restrict sensory function and visual information. To identify the bottlenecks and guide prosthesis designing, psychophysics simulations of a visual prosthesis in normally sighted individuals are desirable. In this study, psychophysical experiments of discriminating objects with similar profiles were used to test the effects of phosphene array parameters (spatial resolution, gray scale, distortion, and dropout rate) on visual information using simulated prosthetic vision. The results showed that the increase in spatial resolution and number of gray levels and the decrease in phosphene distortion and dropout rate improved recognition performance, and the accuracy is 78.5% under the optimum condition (resolution: 32 × 32, gray level: 8, distortion: k = 0, dropout: 0%). In combined parameter tests, significant facial recognition accuracy was achieved for all the images with k = 0.1 distortion and 10% dropout. Compared with other experiments, we find that different objects do not show specific sensitivity to the changes of parameters and visual information is not nearly enough even under the optimum condition. The results suggests that higher spatial resolution and more gray levels are required for visual prosthetic devices and further research on image processing strategies to improve prosthetic vision is necessary, especially when the wearers have to accomplish more than simple visual tasks.     

Complexity Analysis Based on Image-Processing Method and Pixelized Recognition of Chinese Characters Using Simulated Prosthetic Vision

The influence of complexity and minimum resolution necessary for recognition of pixelized Chinese characters (CCs) was investigated by using simulated prosthetic vision. An image-processing method was used to evaluate the complexity of CCs, which is defined as the frequency of black pixels and analyzed by black pixel statistic complexity algorithm. A total of 631 most commonly used CCs that can deliver 80% of the information in Chinese daily reading were chosen as the testing database in order to avoid the negative effect due to illegibility and incognizance. CCs in Hei font style were captured as images and pixelized as 6 × 6, 8 × 8, 10 × 10, and 12 × 12 pixel arrays with square dots. Recognition accuracy of CCs with different complexity and different numbers of pixel arrays was tested by using simulated prosthetic vision. The results indicate that both pixel array number and complexity have significant impact on pixelized reading of CCs. Recognition accuracy of pixelized CCs drops with the increase of complexity and the decrease of pixel number. More than 80% of CCs with any complexity can be recognized correctly; 10 × 10 pixel array can sufficiently provide pixelized reading of CCs for visual prosthesis. Pixelized reading of CCs with low resolution is possible only for characters with low complexity (complexity less than 0.16 for a 6 × 6 pixel array and less than 0.24 for an 8 × 8 pixel array).     

Phosphene Object Perception Employs Holistic Processing During Early Visual Processing Stage

Psychophysical studies have verified the possibility of recovering the visual ability by the form of low‐resolution format of images, that is, phosphene‐based representations. Our previous study has found that early visual processing for phosphene patterns is configuration based. This study further investigated the configural processing mechanisms of prosthetic vision by analyzing the event‐related potential components (P1 and N170) in response to phosphene face and non‐face stimuli. The results reveal that the coarse processing of phosphenes involves phosphene‐specific holistic processing that recovers separated phosphenes into a gestalt; low‐level feature processing of phosphenes is also enhanced compared with that of normal stimuli due to increased contrast borders introduced by phosphenes; while fine processing of phosphene stimuli is impaired reflected by reduced N170 amplitude because of the degraded detailed features in the low‐resolution format representations. Therefore, we suggest that strategies that can facilitate the specific holistic processing stages of prosthetic vision should be considered in order to improve the performance when designing the visual prosthesis system.     

Cement collar and longitudinal groove: the effects on mechanical stability with aseptic loosening in Müller straight-stem implants

Introduction  The Müller straight-stem prosthesis (GS-77), in contrast to the Müller SL prosthesis (SL-88), is equipped with a cement collar and longitudinal grooves. Comparative observation of these prostheses suggests that cement collars protect against rapid subsidence and that longitudinal grooves prevent premature formation of a varus tilting deformity. Method  Because loosening is often observed in the titanium-alloy models of these implants, this retrospective study focused on these models. Differences based on other parameters known to affect mechanical performance were minimized by careful selection: the use of lateralizing implants, patient population characteristics (gender distribution, age, BMI, activity level), and consistency of prosthetic size (10). The comparability of the bending stiffness of both implants in corresponding sizes was verified using finite element method calculations. Results  Up to the time of revision, 17 GS-77 and 33 SL-88 implants were compared for the amounts of subsidence and prosthetic tilting. Standardized follow-up films were contrasted and radiographically measured. At the time of revision, the GS-77 implants showed subsidence of 4.8 (±3.7) mm and the SL-88 implants 8.2 (±5.1) mm (p = 0.012). Leaving apart 3 cases with an extreme situation tilting of the GS-77 implants, with 1.2 (±0.9)°, averaged only two-thirds that of the SL-88 implants, with 1.9 (±1.1(p = 0.032). These results support the hypothesis that the cement collar of the Müller straight-stem prosthesis protects against rapid subsidence occurring with aseptic loosening of the implant. The longitudinal grooves appear to counter the development of varus tilting.     

Osseointegrated Titanium Implants for Limb Prostheses Attachments: Infectious Complications

Background  

The concept of osseointegration involves direct contact between titanium implant and bone. This transcutaneous prosthetic system for amputees is intended to assure stable long-term fixation. Most metal transcutaneous implants have failed, primarily owing to infection.     

A validated subjective rating of display quality: the Maryland Visual Comfort Scale

Background  

Minimally invasive surgery requires high-quality imaging to provide effective visual displays to surgeons. Whereas objective measures—pixels, resolution, display size, contrast ratio—are used to compare imaging systems, there are no tools for assessing the perceptual impact of these physical measures. We developed the “Maryland Visual Comfort Scale” (MVCS) to measure perceptual qualities in relation to an imaging system. We theorize that what the surgeon perceives as a high-quality image can be summarized by a scoring of seven characteristics related to human perception, and that image quality is not homogenous across a video display such that object location impacts perception and display quality.     

Image recognition with a limited number of pixels for visual prostheses design

With the rapid development and crossover among the information science, microelectronics, material science, and biomedical disciplines, the visual prosthesis makes visual reparation possible. Because the number of stimulation electrodes is strictly limited by various complicated factors, it is necessary to determine the minimum visual requirements to achieve useful artificial vision for image recognition. This research has studied how many pixels individual images need to have to be correctly and economically recognized by blind subjects. In order to extract the figure of the image with a limited number of pixels, we have proposed a wavelet‐based image processing methods, and six resolutions (8 × 8, 16 × 16, 24 × 24, 32 × 32, 48 × 48, and 64 × 64) are investigated. Psychophysical experiments have been designed to verify our proposed image processing method and to investigate the recognition accuracy with a limited number of pixels. The results show that the recognition accuracy increases with the number of pixels. The recognition accuracy varied with tested images, when a resolution of 24 × 24 was used: six of the eight image objects were recognized with an accuracy of >50%, and the remaining two of the eight image objects were recognized with an accuracy of <50%. Moreover, when the resolution is more than 32 × 32, the increase of the recognition accuracy is no longer obvious. We also have investigated the impact of different perspectives of the same object to the recognition accuracy. The experiment shows that providing multiview image sequences, subjects can receive more visual information to obtain higher recognition accuracy.     

Configuration‐Based Processing of Phosphene Pattern Recognition for Simulated Prosthetic Vision

Visual prosthesis can elicit phosphenes by stimulating the retina, optic nerve, or visual cortex along the visual pathway. Psychophysical studies have demonstrated that visual function can be partly recovered with phosphene‐based prosthetic vision. This study investigated the cognitive process of prosthetic vision through a face recognition task. Both behavioral response and the face‐specific N170 component of event‐related potential were analyzed in the presence of face and non‐face stimuli with natural and simulated prosthetic vision. Our results showed that: (i) the accuracy of phosphene face recognition was comparable with that of the normal one when phosphene grid increased to 25 × 21 or more; (ii) shorter response time was needed for phosphene face recognition; and (iii) the N170 component was delayed and enhanced under phosphene stimuli. It was suggested that recognition of phosphene patterns employ a configuration‐based holistic processing mechanism with a distinct substage unspecific to faces.     

Solid-state semiconductors are better alternatives to arc-lamps for efficient and uniform illumination in minimal access surgery

Introduction  Current arc-lamp illumination systems have a number of technical and ergonomic limitations. White light-emitting diodes (LEDs) are energy-efficient solid-state lighting devices which are small, durable and inexpensive. Their use as an alternative to arc-lamp light sources in minimal access surgery has not been explored. This study aims to develop an LED-based endo-illuminator and to determine its lighting characteristics for use in minimal access surgery. Methods  We developed an LED endo-illuminator using a white LED mounted at the tip of a steel rod. Offline image analysis was carried out to compare the illuminated field using the LED endo-illuminator or an arc-lamp based endoscope in terms of uniformity, shadow sharpness and overall image intensity. Direct radiometric power measurements in light intensity and stability were obtained. Visual perception of fine details at the peripheral endoscopic field was assessed by 13 subjects using the different illumination systems. Results  Illumination from the LED endo-illuminator was more uniform compared to illumination from an arc-lamp source, especially at the closer distance of 4 cm (0.0006 versus 0.0028 arbitrary units – lower value indicates more uniform illumination). The shadows were also sharper (edge widths of 16 versus 44 pixels for the first edge and 15 versus 61 pixels for the second edge). The overall mean image intensity was higher (127 versus 100 arbitrary units) when using the autoshutter mode despite the lower direct radiometric power, about one tenth of the arc-lamp endoscopic system. The illumination was also more stable with less flickering (0.02% versus 5% of total power in non-DC components). Higher median scores on visual perception was also obtained (237 versus 157, p < 0.001). Conclusion  The LED endo-illuminator provides more uniform illumination with sharper shadows, less flickering and better illumination for visual perception than the arc-lamp-based system currently used.     

Does Acetabular Retroversion Affect Range of Motion after Total Hip Arthroplasty?

Background  

Increasingly, acetabular retroversion is recognized in patients undergoing hip arthroplasty. Although prosthetic component positioning is not determined solely by native acetabular anatomy, acetabular retroversion presents a dilemma for component positioning if the surgeon implants the device in the anatomic position.     

Serum metal ion levels after rotating-hinge knee arthroplasty: comparison between a standard device and a megaprosthesis

Purpose  

The effects of systemic metal ion exposure in patients with implants made of common prosthetic alloys continue to be a matter of concern. The aim of the study was to determine the measurement values of cobalt (Co), chromium (Cr) and molybdenum (Mo) in serum following rotating-hinge knee arthroplasty.     

Image Processing Strategies Based on a Visual Saliency Model for Object Recognition Under Simulated Prosthetic Vision

Retinal prostheses have the potential to restore partial vision. Object recognition in scenes of daily life is one of the essential tasks for implant wearers. Still limited by the low‐resolution visual percepts provided by retinal prostheses, it is important to investigate and apply image processing methods to convey more useful visual information to the wearers. We proposed two image processing strategies based on Itti's visual saliency map, region of interest (ROI) extraction, and image segmentation. Itti's saliency model generated a saliency map from the original image, in which salient regions were grouped into ROI by the fuzzy c‐means clustering. Then Grabcut generated a proto‐object from the ROI labeled image which was recombined with background and enhanced in two ways—8‐4 separated pixelization (8‐4 SP) and background edge extraction (BEE). Results showed that both 8‐4 SP and BEE had significantly higher recognition accuracy in comparison with direct pixelization (DP). Each saliency‐based image processing strategy was subject to the performance of image segmentation. Under good and perfect segmentation conditions, BEE and 8‐4 SP obtained noticeably higher recognition accuracy than DP, and under bad segmentation condition, only BEE boosted the performance. The application of saliency‐based image processing strategies was verified to be beneficial to object recognition in daily scenes under simulated prosthetic vision. They are hoped to help the development of the image processing module for future retinal prostheses, and thus provide more benefit for the patients.     

Estimation of simulated phosphene size based on tactile perception

Clinical trials have successfully shown that a visual prosthesis can elicit visual perception (phosphenes) in the visual field. Psychophysical studies based on simulated prosthetic vision offer an effective means to evaluate and refine prosthetic vision. We designed three experiments to examine the effect of phosphene luminance, flicker rate, and eccentricity on the ability to estimate simulated phosphene sizes using tactile perception. Thirty subjects participated in the three experiments. There was a linear increase in reported size as visual stimulus size increased. Judgment was significantly affected by stimulus luminance and eccentricity (P < 0.05) but not by flicker rates. Brighter stimuli were perceived as being larger, and the more eccentric the position, the larger the estimated size. These simulation studies, although idealized, suggested that tactile perception is a potential way to estimate phosphene sizes.     

Object Recognition Under Distorted Prosthetic Vision

Psychophysical studies have reported the efficacy of phosphene‐based prosthetic vision in partly recovering the visual function of blind individuals. However, results by far have been based on evenly aligned phosphene arrays, which neglected the complicated visuotopy in the visual prosthesis system. In this study, we investigated how the objects were recognized under the stimuli with distorted phosphene arrays simulated by transformations of barrel distortion, rotation, or translation. The results revealed that distortions significantly decreased the accuracy of categorization (CA) and showed distinct interactive effects with the factors of object category and phosphene array density. Moreover, the CA changed differently with the increase of distortion levels. Regression analysis suggested a phosphene array of at least 10 × 10 be the essential for achieving a CA over the threshold value (CA_t = 62.5%) under distorted prosthetic vision. It is recommended that discriminative features be extracted to improve the performance of prosthetic vision.     

Recognition of Objects in Simulated Irregular Phosphene Maps for an Epiretinal Prosthesis

Visual prostheses offer a possibility of restoring vision to the blind. It is necessary to determine minimum requirements for daily visual tasks. To investigate the recognition of common objects in daily life based on the simulated irregular phosphene maps, the effect of four parameters (resolution, distortion, dropout percentage, and gray scale) on object recognition was investigated. The results showed that object recognition accuracy significantly increased with an increase of resolution. Distortion and dropout percentage had significant impact on the object recognition; with the increase of distortion level and dropout percentage, the recognition decreased considerably. The accuracy decreased significantly only at gray level 2, whereas the other three gray levels showed no obvious difference. The two image processing methods (merging pixels to lower the resolution and edge extraction before lowering resolution) showed significant difference on the object recognition when there was a high degree of distortion level or dot dropout.     

Emerging Ideas: Interleukin-12 Nanocoatings Prevent Open Fracture-associated Infections

Background  

Infection is a major clinical complication of orthopaedic implants and prosthetic devices, and patients with traumatic open fractures have a high risk of infection that may exceed 30%. Surgical trauma, burns, and major injuries such as traumatic open fractures induce immunosuppression, decrease resistance to infection, and decrease production of T helper type 1 (Th1) cytokines.     

Management of delayed capsular hematoma after breast reconstruction

Delayed hematoma associated with breast implants, whether for reconstructive or aesthetic purposes, is a rare entity. Most reported cases have been intracapsular. Although several mechanisms for delayed hematoma have been proposed, its exact etiology and mechanism of formation are not yet fully understood. The authors present a unique case of hematoma formation 5 years after breast reconstruction with a textured anatomic cohesive gel breast implant. The patient experienced severe thrombocytopenia secondary to chemotherapy, which most probably was the triggering etiologic factor. The hematoma was both intra- and extracapsular. The authors speculate that the retrocapsular hematoma component was due to firm adhesion of the textured prosthetic membrane to the posterior capsule and propose that for breast reconstruction with prosthetic implants in patients likely to receive chemotherapy, a smooth implant might be better indicated.     

Postural mechatronic assistant for laparoscopic solo surgery (PMASS)

Background and purpose  Laparoscopes used in laparoscopic surgery are manipulated by human means, passive systems or robotic systems. All three methods accumulate downtime when the laparoscope is cleaned and the optical perspective is adjusted. This work proposes a new navigation system that autonomously handles the laparoscope, with a view to reducing latency, and that allows real-time adjustment of the visual perspective. Methods  The system designed is an intuitive mechatronic system with three degrees of freedom and a single active articulation. The system uses the point of insertion as the invariant point for navigation and has a work space that closely resembles an inverted cone. Results  The mechatronic system has been tested in a physical trainer, cutting and suturing chicken parts, as well as in laparoscopic ovariohysterectomies in dogs and pediatric surgeries. In all the procedures, surgeons were able to auto-navigate and there was no visual tremor while using the system. Surgeons performed visual approaches in real time and had both hands free to carry out the procedure. Conclusion  This new mechatronic system allows surgeons to perform solo surgery. Cleaning and positioning downtime are reduced, since it is the surgeon him/herself who handles the optics and selects the best visual perspective for the surgery.