Early experience with a patient‐specific virtual surgical simulation for rehearsal of endoscopic skull‐base surgery

Background

With the help of contemporary computer technology it is possible to create a virtual surgical environment (VSE) for training. This article describes a patient‐specific virtual rhinologic surgical simulation platform that supports rehearsal of endoscopic skull‐base surgery. We also share our early experience with select cases.

Methods

A rhinologic VSE was developed, featuring a highly efficient direct 3‐dimensional (3D) volume renderer with simultaneous stereoscopic feedback during surgical manipulation of the virtual anatomy, as well as high‐fidelity haptic feedback. We conducted a retrospective analysis on 10 patients who underwent various forms of sinus and ventral skull‐base surgery to assess the ability of the rhinologic VSE to replicate actual intraoperative findings.

Results

In all 10 cases, the simulation experience was realistic enough to perform dissections in a similar manner as in the actual surgery. Excellent correlation was found in terms of surgical exposure, anatomical features, and the locations of pathology.

Conclusion

The current rhinologic VSE shows sufficient realism to allow patient‐specific surgical rehearsal of the sinus and ventral skull base. Further validation studies are needed to assess the benefits of performing patient‐specific rehearsal.     

Enhanced effectiveness in visuo‐haptic object‐selective brain regions with increasing stimulus salience

The occipital and parietal lobes contain regions that are recruited for both visual and haptic object processing. The purpose of the present study was to characterize the underlying neural mechanisms for bimodal integration of vision and haptics in these visuo‐haptic object‐selective brain regions to find out whether these brain regions are sites of neuronal or areal convergence. Our sensory conditions consisted of visual‐only (V), haptic‐only (H), and visuo‐haptic (VH), which allowed us to evaluate integration using the superadditivity metric. We also presented each stimulus condition at two different levels of signal‐to‐noise ratio or salience. The salience manipulation allowed us to assess integration using the rule of inverse effectiveness. We were able to localize previously described visuo‐haptic object‐selective regions in the lateral occipital cortex (lateral occipital tactile‐visual area) and the intraparietal sulcus, and also localized a new region in the left anterior fusiform gyrus. There was no evidence of superadditivity with the VH stimulus at either level of salience in any of the regions. There was, however, a strong effect of salience on multisensory enhancement: the response to the VH stimulus was more enhanced at higher salience across all regions. In other words, the regions showed enhanced integration of the VH stimulus with increasing effectiveness of the unisensory stimuli. We called the effect “enhanced effectiveness.” The presence of enhanced effectiveness in visuo‐haptic object‐selective brain regions demonstrates neuronal convergence of visual and haptic sensory inputs for the purpose of processing object shape. Hum Brain Mapp, 2010. © 2009 Wiley‐Liss, Inc.     

Reviewing the technological challenges associated with the development of a laparoscopic palpation device

Minimally invasive surgery (MIS) has heralded a revolution in surgical practice, with numerous advantages over open surgery. Nevertheless, it prevents the surgeon from directly touching and manipulating tissue and therefore severely restricts the use of valuable techniques such as palpation. Accordingly a key challenge in MIS is to restore haptic feedback to the surgeon. This paper reviews the state‐of‐the‐art in laparoscopic palpation devices (LPDs) with particular focus on device mechanisms, sensors and data analysis. It concludes by examining the challenges that must be overcome to create effective LPD systems that measure and display haptic information to the surgeon for improved intraoperative assessment. Copyright © 2012 John Wiley & Sons, Ltd.     

Direct coupling of haptic signals between hands

Although motor actions can profoundly affect the perceptual interpretation of sensory inputs, it is not known whether the combination of sensory and movement signals occurs only for sensory surfaces undergoing movement or whether it is a more general phenomenon. In the haptic modality, the independent movement of multiple sensory surfaces poses a challenge to the nervous system when combining the tactile and kinesthetic signals into a coherent percept. When exploring a stationary object, the tactile and kinesthetic signals come from the same hand. Here we probe the internal structure of haptic combination by directing the two signal streams to separate hands: one hand moves but receives no tactile stimulation, while the other hand feels the consequences of the first hand’s movement but remains still. We find that both discrete and continuous tactile and kinesthetic signals are combined as if they came from the same hand. This combination proceeds by direct coupling or transfer of the kinesthetic signal from the moving to the feeling hand, rather than assuming the displacement of a mediating object. The combination of signals is due to perception rather than inference, because a small temporal offset between the signals significantly degrades performance. These results suggest that the brain simplifies the complex coordinate transformation task of remapping sensory inputs to take into account the movements of multiple body parts in haptic perception, and they show that the effects of action are not limited to moving sensors.Motor and kinesthetic signals arising from the movement of the eyes in the head, and translation of the eyes and ears in space due to head and body movements, have been shown to play an important role in visual (1–3) and auditory (4–6) perception. However, because of the small number of sensory surfaces in these modalities, and the rigid constraints on their movement, the number of kinesthetic degrees of freedom is limited. In active touch or the haptic modality (7–13), the large number of sensory surfaces, and the nearly unlimited ways these surfaces can move, lead to the question of how movement can be represented and associated with the cutaneous or tactile signals. To study how tactile and kinesthetic cues are combined to haptically perceive object shape and size, we created a novel haptic stimulus in which these cues were completely dissociated. This stimulus consisted of simulated triangles felt through a narrow slit, as in anorthoscopic perception in vision (14–16) or haptic perception (17), and as illustrated in Fig. 1.Open in a separate windowFig. 1.Tactile stimuli and movement conditions. (A) An example of the tactile stimulus, here an expanding bar. The stimulus is shown as a series of “snapshots,” with the time running to the Right—the actual stimulus was continuous. The red rectangle represents the vibrating pins. (B) SAME condition: the same hand moves and experiences the tactile consequences of the movement. The hand moves the tactile display (green rectangle) mounted on the slider. In this example, a forward movement (away from the participant, upward in the figure) together with an expanding bar simulates a backward-pointing triangle (shown in red), felt through a slit. (C) DIFF condition: one hand moves, while the other hand, immobile, receives the tactile stimulus. The tactile signal (expanding bar) and kinesthetic signal (forward movement) are the same as in the previous example. Will the observer perceive simply an expanding bar or triangle in space? If a triangle is perceived, in which direction will it point? (D) IMMOB condition: the tactile signal is presented alone, with no movement. The expanding or contracting bar’s width as a function of time is a replay of a previous trial. (E) Correct spatial orientation of the triangle in the SAME condition, as a function of the movement direction (forward or backward) and the tactile stimulus (expansion or contraction). This truth table is an exclusive-or function, which has null correlations with its two input signals. Using either of the two signals alone will result in chance performance.The tactile signal consisted of a line that expanded or contracted on the index finger, delivered using a tactile display composed of pins that could vibrate independently, as shown in Fig. 1A. In one condition (SAME), the display was mounted on a slider that experimental participants slid along a track perpendicular to the tactile expansion or contraction, as shown in Fig. 1B. The slider’s position was used to update the tactile display to simulate stationary triangles of various lengths, oriented toward or away from the participant, felt through a virtual slit that moved with the finger. Would participants perceive an extended triangular shape, rather than the proximal stimulus consisting of expansion or contraction, and if so, which orientation would they perceive? For example, a backward-pointing triangle could result from a tactile expansion coupled with a forward movement (as shown in Fig. 1B, where forward motions and orientations—away from the participant—are shown as upward), or a contraction coupled with a backward movement—and vice versa for a forward-pointing triangle (Fig. 1E). Thus, the directions of the tactile stimulus and of the finger movement are insufficient by themselves to yield veridical perception of triangle orientation, and must be combined in an exclusive-or function (Fig. 1E) to yield veridical perception of object orientation. There are no purely tactile shape cues: the triangle’s edges are not slanted in our tactile stimulus (in contrast to the edges of an actual triangle behind a slit). Therefore, from the functional point of view, the only way to obtain triangle orientation is for the nervous system to make use of the exclusive-or rule on some level. Below we will show that this rule is applied perceptually rather than through cognitive inference.Crucially, we separated tactile and kinesthetic stimuli in the different (DIFF) condition. Participants moved the slider with one hand, and received the tactile stimulation through the stationary fingertip of the other hand (Fig. 1C). The position of the moving was measured and used to update the tactile display as in the SAME condition. Finally, in the immobile (IMMOB) condition, neither hand moved, and the durations of the tactile expansions and contractions were the same as in the other conditions (Fig. 1D). Thus, in both the DIFF and IMMOB conditions, we simulated a moving triangle felt through a stationary slit. In all conditions, participants reported both the perceived orientation and size of the triangle using a visual probe at the end of each trial. Participants closed their eyes during the movement and tactile stimulation, and therefore could not see the display during the crucial part of each trial. In the movement conditions (SAME and DIFF) the direction and speed of finger movement varied from trial to trial. Each of the three conditions was performed in separate blocks for different hands, with the blocks in random order.     

Preliminary testing by adults of a haptics-assisted robot platform designed for children with physical impairments to access play

ABSTRACT

Development of children’s cognitive and perceptual skills depends heavily on object exploration and experience in their physical world. For children who have severe physical impairments, one of the biggest concerns is the loss of opportunities for meaningful play with objects, including physical contact and manipulation. Assistive robots can enable children to perform object manipulation through the control of simple interfaces. Touch sensations conveyed through haptic interfaces in the form of force reflection or force assistance can help a child to sense the environment and to control a robot. A robotic system with forbidden region virtual fixtures (VFs) was tested in an object sorting task. Three sorting tasks—by color, by shape, and by both color and shape—were performed by 10 adults without disability and one adult with cerebral palsy. Tasks performed with VFs were accomplished faster than tasks performed without VFs, and deviations of the motion area were smaller with VFs than without VFs. For the participant with physical impairments, two out of three tasks were slower with the VFs. This implies that forbidden region VFs are not always able to improve user task performance. Alignment with an individual’s unique motion characteristics can improve VF assistance.     

Improving the human–robot interface for telemanipulated robotic long bone fracture reduction: Joystick device vs. haptic manipulator

Objectives

Intramedullary nailing is the treatment of choice for femoral shaft fractures. However, there are several problems associated with the technique, e.g. high radiation exposure and rotational malalignment. Experimental robotic assistance has been introduced to improve the quality of the reduction and to reduce the incidence of rotational malalignment. In the current study, we compare two devices for control of the fracture fragments during telemanipulated reduction.

Methods

Ten male and ten female subjects were asked to participate as examiners in this experiment. A computer program was developed to render and manipulate CT‐based renderings of femur fracture bone fragments. The user could manipulate the fragments using either a simple joystick device or a haptic manipulator. Each examiner performed telemanipulated reduction of 10 virtual fracture models of varying difficulty with each device (five in a ‘training phase’ and five in a ‘testing phase’). Mixed models were used to test whether using the haptic device improved alignment accuracy and improved reduction times compared to using a joystick.

Results

Reduction accuracy was not significantly different between devices in either the training phase or the testing phase (P > 0.05). Reduction time was significantly higher for the Phantom device than for the Joystick in the training phase (P < 0.0001), but it was no different in the testing phase (P = 0.865). High spatial ability with electronics had a significant effect on the alignment of fracture reduction and time to reduction.

Conclusions

The Joystick and the Phantom devices resulted in similarly accurate reductions, with the Joystick having an easier learning curve. The Phantom device offered no advantage over the Joystick for fracture telemanipulation. Considering the high cost of the Phantom device and the lack of a demonstrable advantage over the Joystick, its use is not justified for implementation in a fracture telemanipulation workflow. The Joystick remains as a low‐cost and effective device for developing 3D fracture telemanipulation techniques.     

EEG oscillations during caress‐like affective haptic elicitation

Noninvasive, objective quantitative techniques to gauge emotional states are fundamental for clinical psychology as they overcome subjective bias of currently used questionnaires. To this end, we investigated brain oscillatory EEG activity during caresslike, affective haptic elicitation conveyed on the forearm at two force (strength of the caress) and three velocity (velocity of the caress) levels. Thirty‐two healthy subjects (16 female) were asked to assess each haptic stimulus in terms of arousal (i.e., intensity of emotional perception) and valence (i.e., pleasantness/unpleasantness of emotional perception) scores, according to the circumplex model of affect. Changes in brain oscillations were quantified through spectral and functional connectivity analyses. EEG power spectra were estimated through the individual α frequency peak. Results, expressed in terms of p‐value topographic maps, revealed a suppression of α, β, and γ oscillations over the contralateral somatosensory cortex during unpleasant caresses performed with the lowest force (2 N) and the highest velocity (65 mm/s). Conversely, pleasant caresses at the highest force were associated with a significant decrease of EEG oscillatory activity over the midfrontal region, at frequency bands including α, β, and γ. A correlation analysis showed that EEG γ power from the somatosensory area was linked to caressing force. The more unpleasant the affective cutaneous stimuli, the more the brain dynamics decrease in activity all over the scalp, primarily showing a suppression of α power over the midfrontal cortex. These results also pave the way for the design of haptic systems eliciting a given emotional state.     

Virtual reality and haptics in dental education: Implementation progress and lessons learned after a decade

Introduction

The search for alternative training environments in dentistry responded mainly to scarcity and lack of standardisation of training material and non-availability of specific clinical procedures. The development of haptic virtual reality (VR) dental trainers provides a platform where irreversible procedures can be safely and unlimitedly practised. The aim of this study was to assess the educational implementation of these devices and evaluate schools' satisfaction.

Methods

Dental schools that were using haptic VR dental trainers, were approached. The Dental Trainer User Inventory (DTUI), addressing the educational implementation and users' satisfaction, was developed and distributed.

Results

Twenty-seven schools completed the DTUI. The total number of VR dental trainers available varied from one to 42 devices with a mean of 7 devices. The dental trainer was mostly made available from the first year (63.0%) of the undergraduate program, but it was mostly integrated into the curriculum by the third year (70.4%). Curricular integration was reported by 18 schools (66.7%), while nine schools (33.3%) indicated that they had not yet achieved integration. Twenty-one schools (69.4%) were ‘satisfied’ or ‘very satisfied’ with the devices, while two schools (7.4%) were dissatisfied and six schools (22.2%) were neither satisfied nor dissatisfied.

Conclusion

VR haptic dental trainers are implemented in multiple dental programs and are also being used for educational research and clinical training. Even though curricular changes and teachers' acceptance remain a challenge, most schools are satisfied with VR haptic dental trainers and would recommend the device to other schools.