Speech-independent production of communicative gestures: evidence from patients with complete callosal disconnection

Recent neuropsychological, psycholinguistic, and evolutionary theories on language and gesture associate communicative gesture production exclusively with left hemisphere language production. An argument for this approach is the finding that right-handers with left hemisphere language dominance prefer the right hand for communicative gestures. However, several studies have reported distinct patterns of hand preferences for different gesture types, such as deictics, batons, or physiographs, and this calls for an alternative hypothesis. We investigated hand preference and gesture types in spontaneous gesticulation during three semi-standardized interviews of three right-handed patients and one left-handed patient with complete callosal disconnection, all with left hemisphere dominance for praxis. Three of them, with left hemisphere language dominance, exhibited a reliable left-hand preference for spontaneous communicative gestures despite their left hand agraphia and apraxia. The fourth patient, with presumed bihemispheric language representation, revealed a consistent right-hand preference for gestures. All four patients displayed batons, tosses, and shrugs more often with the left hand/shoulder, but exhibited a right hand preference for pantomime gestures. We conclude that the hand preference for certain gesture types cannot be predicted by hemispheric dominance for language or by handedness. We found distinct hand preferences for specific gesture types. This suggests a conceptual specificity of the left and right hand gestures. We propose that left hand gestures are related to specialized right hemisphere functions, such as prosody or emotion, and that they are generated independently of left hemisphere language production. Our findings challenge the traditional neuropsychological and psycholinguistic view on communicative gesture production.     

Deictic gestures and symbolic gestures produced by adults in an experimental context: Hand shapes and hand preferences

The objective of this study was to gain new insights into the processes underlying gestural communication in adults by examining hand shapes and hand preference patterns associated with different types of gestures. Several communicative situations eliciting pointing gestures and symbolic gestures were presented to 81 participants in an experimental context. Results have highlighted some differences in hand shapes depending on the function of pointing: contrary to results reported in children, the proportion of index-finger gestures was higher in imperative situations than in declarative situations. The distance between the gesturer and the referent was also found to influence hand shapes, proximal pointing being more frequently associated with index-finger gestures than distal pointing. The comparison of hand preferences revealed a greater right-sided asymmetry for declarative pointing than for non-communicative activities, whereas there was no difference between imperative pointing and non-communicative activities, or between symbolic gestures and non-communicative activities. The present study thus sheds some light on the features and the roles of communicative gestures in adults. Results are discussed in relation to the ontogenetic and phylogenetic origins of communication.     

Monkey drumming reveals common networks for perceiving vocal and nonvocal communication sounds

Salient sounds such as those created by drumming can serve as means of nonvocal acoustic communication in addition to vocal sounds. Despite the ubiquity of drumming across human cultures, its origins and the brain regions specialized in processing such signals remain unexplored. Here, we report that an important animal model for vocal communication, the macaque monkey, also displays drumming behavior, and we exploit this finding to show that vocal and nonvocal communication sounds are represented by overlapping networks in the brain''s temporal lobe. Observing social macaque groups, we found that these animals use artificial objects to produce salient periodic sounds, similar to acoustic gestures. Behavioral tests confirmed that these drumming sounds attract the attention of listening monkeys similarly as conspecific vocalizations. Furthermore, in a preferential looking experiment, drumming sounds influenced the way monkeys viewed their conspecifics, suggesting that drumming serves as a multimodal signal of social dominance. Finally, by using high-resolution functional imaging we identified those brain regions preferentially activated by drumming sounds or by vocalizations and found that the representations of both these communication sounds overlap in caudal auditory cortex and the amygdala. The similar behavioral responses to drumming and vocal sounds, and their shared neural representation, suggest a common origin of primate vocal and nonvocal communication systems and support the notion of a gestural origin of speech and music.     

Enabling devices,empowering people: The design and evaluation of Trackball EdgeWrite

Purpose. To describe the research and development that led to Trackball EdgeWrite, a gestural text entry method that improves desktop input for some people with motor impairments. To compare the character-level version of this technique with a new word-level version. Further, to compare the technique with competitor techniques that use on-screen keyboards.

Method. A rapid and iterative design-and-test approach was used to generate working prototypes and elicit quantitative and qualitative feedback from a veteran trackball user. In addition, theoretical modelling based on the Steering law was used to compare competing designs.

Results. One result is a refined software artifact, Trackball EdgeWrite, which represents the outcome of this investigation. A theoretical result shows the speed benefit of word-level stroking compared to character-level stroking, which resulted in a 45.0% improvement. Empirical results of a trackball user with a spinal cord injury indicate a peak performance of 8.25 wpm with the character-level version of Trackball EdgeWrite and 12.09 wpm with the word-level version, a 46.5% improvement. Log file analysis of extended real-world text entry shows stroke savings of 43.9% with the word-level version. Both versions of Trackball EdgeWrite were better than on-screen keyboards, particularly regarding user preferences. Follow-up correspondence shows that the veteran trackball user with a spinal cord injury still uses Trackball EdgeWrite on a daily basis 2 years after his initial exposure to the software.

Conclusions. Trackball EdgeWrite is a successful new method for desktop text entry and may have further implications for able-bodied users of mobile technologies. Theoretical modelling is useful in combination with empirical testing to explore design alternatives. Single-user lab and field studies can be useful for driving a rapid iterative cycle of innovation and development.     

Use of co-verbal gestures during word-finding difficulty among Cantonese speakers with fluent aphasia and unimpaired controls

Background: Co-verbal gestures refer to hand or arm movements made during speaking. Spoken language and gestures have been shown to be tightly integrated in human communication.

Aims: The present study investigated whether co-verbal gesture use was associated with lexical retrieval in connected speech in unimpaired speakers and persons with aphasia (PWA).

Methods & Procedures: Narrative samples of 58 fluent PWA and 58 control speakers were extracted from Cantonese AphasiaBank. Based on the indicators of word-finding difficulty (WFD) in connected speech adapted from previous research and a gesture annotation system with independent coding of gesture forms and functions, all WFD instances were identified. The presence and type of gestures accompanying each incident of WFD were then annotated. Finally, whether the use of gesture was accompanied by resolution of WFD (i.e., the corresponding target word could be retrieved) was examined.

Outcomes & Results: Employment of co-verbal gesture did not seem to be related to the success of word retrieval. PWA’s naming ability at single-word level and their overall language ability (as reflected by the aphasia quotient of the Cantonese version of the Western Aphasia Battery) were found to be the two strongest predictors of success rate of resolving WFD.

Conclusions: The Lexical Retrieval Hypothesis highlighting the facilitative functions of iconic and metaphoric gestures in lexical retrieval was not supported. Challenges in conducting research related to WFD, and the clinical implications in gesture-based language intervention for PWA were discussed.     

Impaired gesture performance in schizophrenia: Particular vulnerability of meaningless pantomimes

Schizophrenia patients frequently present with subtle motor impairments, including higher order motor function such as hand gesture performance. Using cut off scores from a standardized gesture test, we previously reported gesture deficits in 40% of schizophrenia patients irrespective of the gesture content. However, these findings were based on normative data from an older control group. Hence, we now aimed at determining cut-off scores in an age and gender matched control group. Furthermore, we wanted to explore whether gesture categories are differentially affected in Schizophrenia. Gesture performance data of 30 schizophrenia patients and data from 30 matched controls were compared. Categories included meaningless, intransitive (communicative) and transitive (object related) hand gestures, which were either imitated or pantomimed, i.e. produced on verbal command. Cut-off scores of the age matched control group were higher than the previous cut-off scores in an older control group. An ANOVA tested effects of group, domain (imitation or pantomime), and semantic category (meaningless, transitive or intransitive), as well as their interaction. According to the new cut-off scores, 67% of the schizophrenia patients demonstrated gestural deficits. Patients performed worse in all gesture categories, however meaningless gestures on verbal command were particularly impaired (p=0.008). This category correlated with poor frontal lobe function (p<0.001).     

Examination of the relationship between gestures and vocabulary in children with autism spectrum disorder at different language stages

In this study, the relationship between vocabulary and gesture use is examined in children with autism spectrum disorder (ASD) at different language stages. A total of 72 children with ASD between the ages of 3.0 and 8.2 years participated in the study. Gesture use was assessed by using observation-based procedures, while the number of different words (NDW) and the mean length of utterance (MLU) was derived from conversational language samples. Significant differences were found in all gesture kinds among different language stages, in children with ASD. It was found that declarative deictic gestures and conventional/pantomime gestures meaningfully predict the NDW, while imperative deictic gestures do not. According to the findings of this study, gestures show different developmental features at different stages of language development in children with ASD. The findings show that declarative deictic gestures and conventional/pantomime gestures may play a significant role in vocabulary development in children with ASD.     

Neural Basis of Speech-Gesture Mismatch Detection in Schizophrenia Spectrum Disorders

Patients with schizophrenia spectrum disorders (SSD) exhibit an aberrant perception and comprehension of abstract speech-gesture combinations associated with dysfunctional activation of the left inferior frontal gyrus (IFG). Recently, a significant deficit of speech-gesture mismatch detection was identified in SSD, but the underlying neural mechanisms have not yet been examined. A novel mismatch-detection fMRI paradigm was implemented manipulating speech-gesture abstractness (abstract/concrete) and relatedness (related/unrelated). During fMRI data acquisition, 42 SSD patients (schizophrenia, schizoaffective disorder, or other non-organic psychotic disorder [ICD-10: F20, F25, F28; DSM-IV: 295.X]) and 36 healthy controls were presented with short video clips of an actor reciting abstract or concrete sentences accompanied by either a semantically related or unrelated gesture. Participants indicated via button press whether they perceived each gesture as matching the speech content or not. Speech-gesture mismatch detection performance was significantly impaired in patients compared to controls. fMRI data analysis revealed that patients showed lower activation in bilateral frontal areas, including the IFG for all abstract > concrete speech-gesture pairs. In addition, they exhibited reduced engagement of the right supplementary motor area (SMA) and bilateral anterior cingulate cortices (ACC) for unrelated > related stimuli. We provide first evidence that impaired speech-gesture mismatch detection in SSD could be the result of dysfunctional activation of the SMA and ACC. Failure to activate the left IFG disrupts the integration of abstract speech-gesture combinations in particular. Future investigations should focus on brain stimulation of the SMA, ACC, and the IFG to improve communication and social functioning in SSD.     

Clusters of nonverbal behavior differentiate truths and lies about future malicious intent in checkpoint screening interviews

Recent research has shown that nonverbal behavior (NVB) assessed across multiple channels can differentiate truthtellers from liars. No study, however, has examined whether or not multiple NVBs can differentiate truths from lies about intent regarding future malicious behavior, or across multiple cultural/ethnic groups. We address this gap by examining truths and lies about intent to commit a malicious act in the future in brief, checkpoint-type security screening interviews. Data from four NVB channels producing twenty-one observable NVBs were coded and analyzed using different analytic strategies. Clusters of NVB were found to differentiate truthtellers from liars at statistically significant levels, and substantially beyond the ability of human observers. The findings showed that clusters of NVB can differentiate truthtellers from liars even in brief, checkpoint-type interviews.     

A multifactorial investigation of captive gorillas’ intraspecific gestural laterality

Multifactorial investigations of intraspecific laterality of primates’ gestural communication aim to shed light on factors that underlie the evolutionary origins of human handedness and language. This study assesses gorillas’ intraspecific gestural laterality considering the effect of various factors related to gestural characteristics, interactional context and sociodemographic characteristics of signaller and recipient. Our question was: which factors influence gorillas’ gestural laterality? We studied laterality in three captive groups of gorillas (N?=?35) focusing on their most frequent gesture types (N?=?16). We show that signallers used predominantly their hand ipsilateral to the recipient for tactile and visual gestures, whatever the emotional context, gesture duration, recipient’s sex or the kin relationship between both interactants, and whether or not a communication tool was used. Signallers’ contralateral hand was not preferentially used in any situation. Signallers’ right-hand use was more pronounced in negative contexts, in short gestures, when signallers were females and its use increased with age. Our findings showed that gorillas’ gestural laterality could be influenced by different types of social pressures thus supporting the theory of the evolution of laterality at the population level. Our study also evidenced that some particular gesture categories are better markers than others of the left-hemisphere language specialization.