基于深浅特征融合的深度卷积残差网络的脑电情绪识别模型

基于脑电信号的智能情绪识别系统具有便携性、高时间分辨率、实时性等特点,能够在健康、娱乐、教育等多个领域实现情绪监控与调节的应用.但由于脑电信号的非平稳性和个体差异性,传统分类器难以深入提取脑电信号中潜在的与情绪语义相关的特征.为了有效地提取脑电特征,提高脑电-情绪识别的准确性,提出一种新型的基于深浅特征融合的深度卷积残...     

国际情绪图片系统在中国的试用研究

目的:对美国NIMH(National Institute of Mental Health)情绪与注意研究中心编制的国际情绪图片系统(IAPS)在中国进行试用研究.方法:男女各24名大学生对330幅图片的愉悦度、唤醒度和优势度三个维度进行了自我报告的9点量表评分.结果:中外被试的三个维度与NIMH结果的相关系数分别为0.913、0.833和0.856,但差异有统计学意义(P<0.01).男女生在三维度上相关系数分别为0.968、0.876和0.945(P均<0.01),其中愉悦度和唤醒度差异有统计学显著性(P<0.01).聚类分析可将图片聚为几类,大致可引发愉快、厌恶、恐惧等情绪.结论:IAPS具有较好的国际通用性,但仍由于文化、个性等因素存在差异,使用前需修订,其图片数据库也有待进一步完善.男女生对部分图片的情绪感受有所不同.聚类分析结果提示可以同时从情绪维度和情绪类型两种途径研究情绪问题,两者结果也有相互比较的可能性.     

Emotion processing in three systems: the medium and the message.

In the context of picture viewing, consistent and specific relationships have been found between two emotion dimensions (valence and arousal) and self-report, physiological and overt behavioral responses. Relationships between stimulus content and the emotion-response profile can also be modulated by the formal properties of stimulus presentation such as screen size. The present experiment explored the impact of another presentation attribute, stimulus motion, on the perceived quality of the induced emotion and on its associated physiological response pattern. Using a within-subject design, moving and still versions of emotion-eliciting stimuli were shown to 35 subjects while facial muscle, heart rate, skin conductance, and emotion self-reports were monitored. The impact of motion was dramatic. Self-report and physiological data suggested strongly that motion increased arousal, had little impact on valence, and captured and sustained the subject's attention to the image.     

Between-person and within-person approaches to the prediction of ambulatory blood pressure: the role of affective valence and intensity

Identifying momentary influences on ambulatory blood pressure (ABP) will help explain ABP variability; however, most research only examines aggregate ABP at the between-person level. This study used within-person methods to examine whether affective dimensions—valence and arousal—differentially predicted momentary ABP levels. A community sample (n = 39) wore an ABP cuff that took BP measurements every 20 min for 24 h. At each measurement, participants reported levels of valence and arousal on electronic diaries. Multilevel modeling was used to examine the effects of momentary and person-averaged levels of valence and arousal on ABP. Greater momentary negative valence and arousal predicted higher systolic BP compared to more positive or lower arousal assessments; higher averaged levels of arousal predicted higher DBP. The results suggest the independence of the effects of valence and arousal on BP. These findings have important implications for designing interventions to lower ABP.     

国际情绪图片系统的本土化研究:在中国大学生群体中的评定

目的:国际情绪图片系统(LAPS)是美国情绪与注意研究中心为情绪相关研究编制的标准化图片系统.但其各项参数尚未在中国人群中得以完整地评定与验证.方法:本研究选取中国大学生被试300名,对全部704张LAPS图片从效价、唤醒度、优势度三个维度进行了中国本土化评定.结果:中国被试评定虽然与LAPS原始评分具有较高的相关,但是在效价与唤醒度维度上均具有显著的差异;对于与中国文化特征相冲突的强烈、直接的情绪图片.中国被试在效价方面的评分显著低于原始结果,而唤醒度高于原始评定.结论:IAPS图片虽然具有较好的国际适用性,但虑及东西方文化差异,中国研究中应参考本土评定结果选取相关的LAPS图片.     

Looking at pictures: Affective, facial, visceral, and behavioral reactions

Colored photographic pictures that varied widely across the affective dimensions of valence (pleasant-unpleasant) and arousal (excited-calm) were each viewed for a 6-s period while facial electromyographic (zygomatic and corrugator muscle activity) and visceral (heart rate and skin conductance) reactions were measured. Judgments relating to pleasure, arousal, interest, and emotional state were measured, as was choice viewing time. Significant covariation was obtained between (a) facial expression and affective valence judgments and (b) skin conductance magnitude and arousal ratings. Interest ratings and viewing time were also associated with arousal. Although differences due to the subject's gender and cognitive style were obtained, affective responses were largely independent of the personality factors investigated. Response specificity, particularly facial expressiveness, supported the view that specific affects have unique patterns of reactivity. The consistency of the dimensional relationships between evaluative judgments (i.e., pleasure and arousal) and physiological response, however, emphasizes that emotion is fundamentally organized by these motivational parameters.     

Modulatory effects of emotion and sleep on recollection and familiarity

Growing evidence suggests that declarative memory benefits from the modulatory effects of emotion and sleep. The primary goal of the present study was to determine whether these two factors interact to enhance memory or they act independently of each other. Twenty-eight volunteers participated in the study. Half of them were sleep deprived the night immediately following the exposure to emotional and non-emotional images, whereas the control group slept at home. Their memory for images was tested 1 week later along the valence and arousal dimension of emotion with the remember–know procedure. As emotional events appear to gain preference during encoding, via the modulatory effect of amygdala on prefrontal and medial temporal lobe regions, conscious retrieval of emotional pictures (relative to neutral ones) was expected to be less disrupted by sleep loss. Results indicated that emotional images were more richly experienced in memory than neutral, particularly those with high arousal and positive valence. Even though sleep deprivation resulted in behavioral impairment at retrieval of both emotional and neutral images, results revealed that remember-based recognition accuracy and its underlying process of recollection for emotional images were less influenced by the lack of sleep (the mean difference between control and sleep-deprived subjects was around 40% higher for neutral images than for emotional images). Familiarity, however, was affected by neither emotion nor sleep. Taken together, these results suggest that emotion and sleep influence differentially the subjective experience of remembering and knowing and the underlying processes of recollection and familiarity through brain mechanisms probably involving amygdala- and hippocampo-neocortical networks respectively.     

Respiratory sinus arrhythmia as an index of emotional response in young adults

The relationship between respiratory sinus arrhythmia (RSA) and valence and arousal remains unclear. In the present study, the associations between emotion responses and tonic or task-related changes in RSA were assessed. Specifically, the sensitivities of changes in interbeat interval, RSA, and skin conductance to the valence and arousal values of emotional stimuli were examined. This study also explored the association between tonic RSA and subjective, expressive, and physiological emotional responses. Response measures were collected from 56 adults during baseline and film-viewing periods. Tonic RSA was not significantly related to any of the response measures. Increased skin conductance and decreased RSA were associated with arousal independent of valence. Interbeat interval was related to affective valence and not arousal. These findings suggest that RSA may be a useful adjunct to skin conductance measures in assessing emotional arousal.     

Anytime multipurpose emotion recognition from EEG data using a Liquid State Machine based framework

Recent technological advances in machine learning offer the possibility of decoding complex datasets and discern latent patterns. In this study, we adopt Liquid State Machines (LSM) to recognize the emotional state of an individual based on EEG data. LSM were applied to a previously validated EEG dataset where subjects view a battery of emotional film clips and then rate their degree of emotion during each film based on valence, arousal, and liking levels. We introduce LSM as a model for an automatic feature extraction and prediction from raw EEG with potential extension to a wider range of applications. We also elaborate on how to exploit the separation property in LSM to build a multipurpose and anytime recognition framework, where we used one trained model to predict valence, arousal and liking levels at different durations of the input. Our simulations showed that the LSM-based framework achieve outstanding results in comparison with other works using different emotion prediction scenarios with cross validation.     

Characterizing emotional response to music in the prefrontal cortex using near infrared spectroscopy

Known to be involved in emotional processing the human prefrontal cortex (PFC), can be non-invasively monitored using near-infrared spectroscopy (NIRS). As such, PFC NIRS can serve as a means for studying emotional processing by the PFC. Identifying patterns associated with emotions in PFC using NIRS may provide a means of bedside emotion identification for nonverbal children and youth with severe physical disabilities. In this study, NIRS was used to characterize the PFC hemodynamic response to emotional arousal and valence in a music-based emotion induction paradigm in 9 individuals without disabilities or known health conditions. In particular, a novel technique based on wavelet-based peak detection was used to characterize chromophore concentration patterns. The maximum wavelet coefficients extracted from oxygenated hemoglobin concentration waveforms from all nine recording locations on the PFC were significantly associated with emotional valence and arousal. Specifically, high arousal and negative emotions were associated with larger maximum wavelet coefficients.     

中国情绪图片系统的编制--在46名中国大学生中的试用

目的：建立本土化的中国情绪图片系统（CAPS）以适应情绪研究的需要。方法：筛选出852幅图片组成CAPS，46名中国大学生对CAPS图片的愉悦度、唤醒度和优势度进行了自我报告的9点量表评分。结果：在CAPS评分中，唤醒度评分结果的一致性最高，愉悦度和优势度评分结果的标准差大于唤醒度的标准差。散点图显示，CAPS在愉悦度和唤醒度上评分分布较为广泛。结论：国际情绪图片系统（IAPS）具有较好的国际通用性，但仍由于文化、个性等因素存在差异，因此编制本土化的中国情绪图片系统是有必要的。     

Affective and physiological responses to environmental noises and music.

Research suggests that respiratory patterns may reflect general dimensions of emotional response. In this study, we investigated the relationships between judgments of affective valence (pleasantness) and arousal and respiratory responses to acoustic stimuli. Sixteen environmental noises and 16 musical fragments of 30 s duration were presented to 31 participants, while respiration, skin conductance level and heart rate were recorded. Judgments of valence and arousal were registered using the 9-point Self-Assessment Manikin. For noises, breathing accelerated and minute ventilation augmented with decreases in pleasantness for low-arousal stimuli and with increases in arousal for positive stimuli. For music, breathing accelerated and minute ventilation augmented with increases both in rated valence and arousal. Skin conductance level increased with arousal ratings for music but not for noises, whereas mean heart rate increased with rated arousal for noises but not for music. Although both noises and music are sound-vibrations, differences in the relationships between affective judgments and physiological responses were found suggesting differences in the processing of the two types of acoustic stimuli.     

Psychophysiological responses as indices of affective dimensions

The startle reflex, facial electromyogram (EMG), and autonomic nervous system responses were examined during imagery varying in affective valence and arousal. Subjects (N= 48) imagined affective situations during tone-cued 8-strials. Startle blink magnitudes were larger and latencies faster during negatively valent than during positively valent conditions and during high-arousal than during low-arousal conditions. Greatest heart rate acceleration and fastest and largest skin conductance responses to startle probes occurred during high-arousal imagery. Zygomatic and orbicularis oculi facial muscle activities were higher during high-arousal imagery, whereas corrugator muscle activity was higher during low-arousal imagery. Zygomatic and corrugator activity also varied with emotional valence. The startle and facial EMG responses are most parsimoniously organized by the negative affect (NA) and positive affect (PA) dimensions, respectively. This NA/PA framework integrates previous research, dimensional theories of emotional behavior, and physiological assessment of pathological emotion.     

The effect of visual sexual content on the event-related potential

In the present study we investigated the effect of emotional valence and arousal value of non-erotic visual stimuli on event-related potentials. We also directly compared the effects of erotic and non-erotic stimuli of comparable valence and arousal levels. Sixteen sexually functional heterosexual males rated 150 computer screen pictures, selected from the International Affective Picture System, representing five stimulus types differing along dimensions of emotional valence and arousal, and erotic vs. non-erotic content. A positive deflection of the ERP between 300 and 500ms after stimulus onset (P300) was found to be sensitive to stimulus differences, with positive valence and high arousal resulting in larger P300. Furthermore, the positive slow wave (PSW) between 500 and 750ms revealed additive effects of valence and arousal value. High arousal and positive valence of the stimuli resulted in larger PSW. Importantly, an interaction effect was absent here. In addition, sexual content led to a larger P300 and PSW than expected on the basis of a mere additive contribution of high arousal and positive valence.     

Cardioceptive accuracy is associated with arousal but not with valence and perceived exertion under physical load

Under resting conditions, cardioceptive accuracy—the acuity of the perception of heartbeats—is associated with the self-reported intensity of affective states but not with reported valence. Physical exertion elicits positive affect below the anaerobic threshold and negative affect above the threshold while arousal gradually increases. The current research aimed to study the associations between cardioceptive accuracy and characteristics of the affective response (arousal and valence) during physical activity. About 67 undergraduate students completed the Schandry task and rated their perceived exertion (Borg-scale) and affective experience (arousal and valence) under three physical loads (running on a treadmill below, around, and above the anaerobic threshold). Cardioceptive accuracy was associated with the arousal component of the affective states during physical activity but not with valence and perceived exertion.     

Effects of unrecognized physiological residual arousal on emotional experience

This study investigated the effect of residual arousal on perceived positive and negative emotion, and the relationship between the valence of emotion and cardiovascular and cardiorespiratory reactivity. A total of 74 participants were randomly assigned to either a Residual Arousal (exercise‐induced) group or a No Residual Arousal (no exercise) group. Following the arousal manipulation, participants watched a video that elicited positive emotion and a video that elicited negative emotion. Within‐person differences revealed greater discrepancies between participants’ reports of positive and negative emotions in response to the videos, indicating that residual arousal caused stronger positive and negative emotions. With regard to physiological reactivity, participants in the No Residual Arousal group exhibited lower heart rate, respiration amplitude, and heart rate variability (LF /HF ratio) during the negative video than during the positive video, suggesting that the positive and negative videos had different influences on physiology. These results support the hypothesis that autonomic activation may be nonspecific with regard to the genesis of emotion, but once a person becomes emotional, physiological reactivity may differ between emotions.     

Biomechanical comparisons of different posterior instrumentation constructs after two-level ALIF: A finite element study

Anterior lumbar interbody fusion (ALIF) with cylindrical cages and supplemental posterior fixation has been widely used for internal disc derangement. However, most researchers have focused on single-level ALIF. Therefore, the biomechanical performance of various fixation constructs after two-level ALIF is not well characterized. This research used three-dimensional finite element models (FEM) with a nonlinear contact analysis to evaluate the initial biomechanical behavior of five types of fixation devices after two-level ALIF (L3/L4, L4/L5) under six loading conditions. These fixation constructs included a three-level pedicle screw and rod, a two-level translaminar facet screw, a two-level transfacet pedicle screw, a bisegmental pedicle screw and rod, and a bisegmental pedicle screw and rod with cross-linking. The FEM's developed in this study demonstrate that, compared to the other four types of posterior fixation constructs analyzed, the three-level pedicle screw and rod provide the best biomechanical stability. Both two-level facet screw fixation constructs showed unfavorable loading in lateral bending. For the construct of the three-level pedicle screw and rod, the middle-segment pedicle screw should not be omitted even though a cross-link is used. The two-level ALIF models with cages and posterior fixation constructs that we developed can be used to evaluate the initial biomechanical performance of a wide variety of posterior fixation devices prior to surgery.     

Subject-independent emotion recognition based on physiological signals: a three-stage decision method

Background

Collaboration between humans and computers has become pervasive and ubiquitous, however current computer systems are limited in that they fail to address the emotional component. An accurate understanding of human emotions is necessary for these computers to trigger proper feedback. Among multiple emotional channels, physiological signals are synchronous with emotional responses; therefore, analyzing physiological changes is a recognized way to estimate human emotions. In this paper, a three-stage decision method is proposed to recognize four emotions based on physiological signals in the multi-subject context. Emotion detection is achieved by using a stage-divided strategy in which each stage deals with a fine-grained goal.

Methods

The decision method consists of three stages. During the training process, the initial stage transforms mixed training subjects to separate groups, thus eliminating the effect of individual differences. The second stage categorizes four emotions into two emotion pools in order to reduce recognition complexity. The third stage trains a classifier based on emotions in each emotion pool. During the testing process, a test case or test trial will be initially classified to a group followed by classification into an emotion pool in the second stage. An emotion will be assigned to the test trial in the final stage. In this paper we consider two different ways of allocating four emotions into two emotion pools. A comparative analysis is also carried out between the proposal and other methods.

Results

An average recognition accuracy of 77.57% was achieved on the recognition of four emotions with the best accuracy of 86.67% to recognize the positive and excited emotion. Using differing ways of allocating four emotions into two emotion pools, we found there is a difference in the effectiveness of a classifier on learning each emotion. When compared to other methods, the proposed method demonstrates a significant improvement in recognizing four emotions in the multi-subject context.

Conclusions

The proposed three-stage decision method solves a crucial issue which is ’individual differences’ in multi-subject emotion recognition and overcomes the suboptimal performance with respect to direct classification of multiple emotions. Our study supports the observation that the proposed method represents a promising methodology for recognizing multiple emotions in the multi-subject context.