Emotion Recognition Based on a EEG–fNIRS Hybrid Brain Network in the Source Space
<b>Background/Objectives</b>: Studies have shown that emotion recognition based on electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) multimodal physiological signals exhibits superior performance compared to that of unimodal approaches. Nonetheless, there remai...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-11-01
|
| Series: | Brain Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3425/14/12/1166 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846105503677022208 |
|---|---|
| author | Mingxing Hou Xueying Zhang Guijun Chen Lixia Huang Ying Sun |
| author_facet | Mingxing Hou Xueying Zhang Guijun Chen Lixia Huang Ying Sun |
| author_sort | Mingxing Hou |
| collection | DOAJ |
| description | <b>Background/Objectives</b>: Studies have shown that emotion recognition based on electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) multimodal physiological signals exhibits superior performance compared to that of unimodal approaches. Nonetheless, there remains a paucity of in-depth investigations analyzing the inherent relationship between EEG and fNIRS and constructing brain networks to improve the performance of emotion recognition. <b>Methods</b>: In this study, we introduce an innovative method to construct hybrid brain networks in the source space based on simultaneous EEG-fNIRS signals for emotion recognition. Specifically, we perform source localization on EEG signals to derive the EEG source signals. Subsequently, causal brain networks are established in the source space by analyzing the Granger causality between the EEG source signals, while coupled brain networks in the source space are formed by assessing the coupling strength between the EEG source signals and the fNIRS signals. The resultant causal brain networks and coupled brain networks are integrated to create hybrid brain networks in the source space, which serve as features for emotion recognition. <b>Results</b>: The effectiveness of our proposed method is validated on multiple emotion datasets. The experimental results indicate that the recognition performance of our approach significantly surpasses that of the baseline method. <b>Conclusions</b>: This work offers a novel perspective on the fusion of EEG and fNIRS signals in an emotion-evoked experimental paradigm and provides a feasible solution for enhancing emotion recognition performance. |
| format | Article |
| id | doaj-art-843c9d13c76e4277a7d73160c725b890 |
| institution | Kabale University |
| issn | 2076-3425 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Brain Sciences |
| spelling | doaj-art-843c9d13c76e4277a7d73160c725b8902024-12-27T14:14:39ZengMDPI AGBrain Sciences2076-34252024-11-011412116610.3390/brainsci14121166Emotion Recognition Based on a EEG–fNIRS Hybrid Brain Network in the Source SpaceMingxing Hou0Xueying Zhang1Guijun Chen2Lixia Huang3Ying Sun4College of Integrated Circuits, Taiyuan University of Technology, Taiyuan 030600, ChinaCollege of Electronic Information Engineering, Taiyuan University of Technology, Taiyuan 030600, ChinaCollege of Electronic Information Engineering, Taiyuan University of Technology, Taiyuan 030600, ChinaCollege of Electronic Information Engineering, Taiyuan University of Technology, Taiyuan 030600, ChinaCollege of Electronic Information Engineering, Taiyuan University of Technology, Taiyuan 030600, China<b>Background/Objectives</b>: Studies have shown that emotion recognition based on electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) multimodal physiological signals exhibits superior performance compared to that of unimodal approaches. Nonetheless, there remains a paucity of in-depth investigations analyzing the inherent relationship between EEG and fNIRS and constructing brain networks to improve the performance of emotion recognition. <b>Methods</b>: In this study, we introduce an innovative method to construct hybrid brain networks in the source space based on simultaneous EEG-fNIRS signals for emotion recognition. Specifically, we perform source localization on EEG signals to derive the EEG source signals. Subsequently, causal brain networks are established in the source space by analyzing the Granger causality between the EEG source signals, while coupled brain networks in the source space are formed by assessing the coupling strength between the EEG source signals and the fNIRS signals. The resultant causal brain networks and coupled brain networks are integrated to create hybrid brain networks in the source space, which serve as features for emotion recognition. <b>Results</b>: The effectiveness of our proposed method is validated on multiple emotion datasets. The experimental results indicate that the recognition performance of our approach significantly surpasses that of the baseline method. <b>Conclusions</b>: This work offers a novel perspective on the fusion of EEG and fNIRS signals in an emotion-evoked experimental paradigm and provides a feasible solution for enhancing emotion recognition performance.https://www.mdpi.com/2076-3425/14/12/1166emotion recognitionEEG–fNIRSsource spacebrain network |
| spellingShingle | Mingxing Hou Xueying Zhang Guijun Chen Lixia Huang Ying Sun Emotion Recognition Based on a EEG–fNIRS Hybrid Brain Network in the Source Space Brain Sciences emotion recognition EEG–fNIRS source space brain network |
| title | Emotion Recognition Based on a EEG–fNIRS Hybrid Brain Network in the Source Space |
| title_full | Emotion Recognition Based on a EEG–fNIRS Hybrid Brain Network in the Source Space |
| title_fullStr | Emotion Recognition Based on a EEG–fNIRS Hybrid Brain Network in the Source Space |
| title_full_unstemmed | Emotion Recognition Based on a EEG–fNIRS Hybrid Brain Network in the Source Space |
| title_short | Emotion Recognition Based on a EEG–fNIRS Hybrid Brain Network in the Source Space |
| title_sort | emotion recognition based on a eeg fnirs hybrid brain network in the source space |
| topic | emotion recognition EEG–fNIRS source space brain network |
| url | https://www.mdpi.com/2076-3425/14/12/1166 |
| work_keys_str_mv | AT mingxinghou emotionrecognitionbasedonaeegfnirshybridbrainnetworkinthesourcespace AT xueyingzhang emotionrecognitionbasedonaeegfnirshybridbrainnetworkinthesourcespace AT guijunchen emotionrecognitionbasedonaeegfnirshybridbrainnetworkinthesourcespace AT lixiahuang emotionrecognitionbasedonaeegfnirshybridbrainnetworkinthesourcespace AT yingsun emotionrecognitionbasedonaeegfnirshybridbrainnetworkinthesourcespace |