HMS-TENet: A hierarchical multi-scale topological enhanced network based on EEG and EOG for driver vigilance estimation
Driving vigilance estimation is an important task for traffic safety. Nowadays, electroencephalography (EEG) and electrooculography (EOG) have made some achievements in vigilance estimation, but there are still some challenges: 1) The traditional approachs with direct multimodal fusion may face the...
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
KeAi Communications Co., Ltd.
2024-12-01
|
| Series: | Biomedical Technology |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2949723X24000333 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846151740004499456 |
|---|---|
| author | Meng Tang Pengrui Li Haokai Zhang Liu Deng Shihong Liu Qingyuan Zheng Hongli Chang Changming Zhao Manqing Wang Guilai Zuo Dongrui Gao |
| author_facet | Meng Tang Pengrui Li Haokai Zhang Liu Deng Shihong Liu Qingyuan Zheng Hongli Chang Changming Zhao Manqing Wang Guilai Zuo Dongrui Gao |
| author_sort | Meng Tang |
| collection | DOAJ |
| description | Driving vigilance estimation is an important task for traffic safety. Nowadays, electroencephalography (EEG) and electrooculography (EOG) have made some achievements in vigilance estimation, but there are still some challenges: 1) The traditional approachs with direct multimodal fusion may face the problems of information redundancy and data dimensionality mismatch; 2) Capture key discriminative features during multimodal fusion without losing specific patterns to each modality. In order to solve the above problems, this paper proposes a approach with fusion of EEG and EOG features in split bands, which not only preserves the information about brain activities in different bands of EEG, but also effectively integrates the relevant information of EOG. On this basis, we further propose a hierarchical multi-scale topological enhanced network (HMS-TENet). This network first introduces a pyramid pooling structure (PPS) to capture contextual relationships from different discriminative perspectives. And then we design a selective convolutional structure (SCS) for adaptive sense-field selection, which enables us to mine the desired discriminative information in small-size features. In addition, we design a topology self-aware attention to enhance the learning of representations of complex topological relationships among EEG channels. Finally, the output of the model can be selected for both regression and classification tasks, providing higher flexibility and adaptability. We demonstrate the robustness, generalizability, and utility of the proposed method based on intra-subject and cross-subject experiments on the SEED-VIG public dataset. Codes are available at https://github.com/tangmeng28/HMS-TENet. |
| format | Article |
| id | doaj-art-1b4982a3e6c64f9489bc29a098757151 |
| institution | Kabale University |
| issn | 2949-723X |
| language | English |
| publishDate | 2024-12-01 |
| publisher | KeAi Communications Co., Ltd. |
| record_format | Article |
| series | Biomedical Technology |
| spelling | doaj-art-1b4982a3e6c64f9489bc29a0987571512024-11-27T05:04:09ZengKeAi Communications Co., Ltd.Biomedical Technology2949-723X2024-12-01892103HMS-TENet: A hierarchical multi-scale topological enhanced network based on EEG and EOG for driver vigilance estimationMeng Tang0Pengrui Li1Haokai Zhang2Liu Deng3Shihong Liu4Qingyuan Zheng5Hongli Chang6Changming Zhao7Manqing Wang8Guilai Zuo9Dongrui Gao10School of Computer Science, Chengdu University of Information Technology, Chengdu, 610225, ChinaSchool of Life Sciences and Technology, University of Electronic Science and Technology of China, Chengdu, 611731, ChinaSchool of Computer Science, Chengdu University of Information Technology, Chengdu, 610225, ChinaSchool of Computer Science, Chengdu University of Information Technology, Chengdu, 610225, ChinaSchool of Computer Science, Chengdu University of Information Technology, Chengdu, 610225, ChinaSchool of Computer Science, Chengdu University of Information Technology, Chengdu, 610225, ChinaShandong Institute of Brain Science and Brain-inspired Research, Shandong First Medical University Shandong Academy of Medical Sciences, J inan, 250117, ChinaSchool of Computer Science, Chengdu University of Information Technology, Chengdu, 610225, ChinaSchool of Computer Science, Chengdu University of Information Technology, Chengdu, 610225, ChinaDepartment of Orthopedic Oncology, the Affiliated Hospital of Qingdao University, No. 59, Haier Road, Qingdao, Shandong, 266000, China; Corresponding author.School of Computer Science, Chengdu University of Information Technology, Chengdu, 610225, China; Sichuan Boruienbrain Technology Co., LT D, Chengdu, 610000, China; Corresponding author. School of Computer Science, Chengdu University of Information Technology, Chengdu, 610225, China.Driving vigilance estimation is an important task for traffic safety. Nowadays, electroencephalography (EEG) and electrooculography (EOG) have made some achievements in vigilance estimation, but there are still some challenges: 1) The traditional approachs with direct multimodal fusion may face the problems of information redundancy and data dimensionality mismatch; 2) Capture key discriminative features during multimodal fusion without losing specific patterns to each modality. In order to solve the above problems, this paper proposes a approach with fusion of EEG and EOG features in split bands, which not only preserves the information about brain activities in different bands of EEG, but also effectively integrates the relevant information of EOG. On this basis, we further propose a hierarchical multi-scale topological enhanced network (HMS-TENet). This network first introduces a pyramid pooling structure (PPS) to capture contextual relationships from different discriminative perspectives. And then we design a selective convolutional structure (SCS) for adaptive sense-field selection, which enables us to mine the desired discriminative information in small-size features. In addition, we design a topology self-aware attention to enhance the learning of representations of complex topological relationships among EEG channels. Finally, the output of the model can be selected for both regression and classification tasks, providing higher flexibility and adaptability. We demonstrate the robustness, generalizability, and utility of the proposed method based on intra-subject and cross-subject experiments on the SEED-VIG public dataset. Codes are available at https://github.com/tangmeng28/HMS-TENet.http://www.sciencedirect.com/science/article/pii/S2949723X24000333VigilanceElectroencephalographyElectrooculographyMulti-taskHMS-TENet |
| spellingShingle | Meng Tang Pengrui Li Haokai Zhang Liu Deng Shihong Liu Qingyuan Zheng Hongli Chang Changming Zhao Manqing Wang Guilai Zuo Dongrui Gao HMS-TENet: A hierarchical multi-scale topological enhanced network based on EEG and EOG for driver vigilance estimation Biomedical Technology Vigilance Electroencephalography Electrooculography Multi-task HMS-TENet |
| title | HMS-TENet: A hierarchical multi-scale topological enhanced network based on EEG and EOG for driver vigilance estimation |
| title_full | HMS-TENet: A hierarchical multi-scale topological enhanced network based on EEG and EOG for driver vigilance estimation |
| title_fullStr | HMS-TENet: A hierarchical multi-scale topological enhanced network based on EEG and EOG for driver vigilance estimation |
| title_full_unstemmed | HMS-TENet: A hierarchical multi-scale topological enhanced network based on EEG and EOG for driver vigilance estimation |
| title_short | HMS-TENet: A hierarchical multi-scale topological enhanced network based on EEG and EOG for driver vigilance estimation |
| title_sort | hms tenet a hierarchical multi scale topological enhanced network based on eeg and eog for driver vigilance estimation |
| topic | Vigilance Electroencephalography Electrooculography Multi-task HMS-TENet |
| url | http://www.sciencedirect.com/science/article/pii/S2949723X24000333 |
| work_keys_str_mv | AT mengtang hmstenetahierarchicalmultiscaletopologicalenhancednetworkbasedoneegandeogfordrivervigilanceestimation AT pengruili hmstenetahierarchicalmultiscaletopologicalenhancednetworkbasedoneegandeogfordrivervigilanceestimation AT haokaizhang hmstenetahierarchicalmultiscaletopologicalenhancednetworkbasedoneegandeogfordrivervigilanceestimation AT liudeng hmstenetahierarchicalmultiscaletopologicalenhancednetworkbasedoneegandeogfordrivervigilanceestimation AT shihongliu hmstenetahierarchicalmultiscaletopologicalenhancednetworkbasedoneegandeogfordrivervigilanceestimation AT qingyuanzheng hmstenetahierarchicalmultiscaletopologicalenhancednetworkbasedoneegandeogfordrivervigilanceestimation AT honglichang hmstenetahierarchicalmultiscaletopologicalenhancednetworkbasedoneegandeogfordrivervigilanceestimation AT changmingzhao hmstenetahierarchicalmultiscaletopologicalenhancednetworkbasedoneegandeogfordrivervigilanceestimation AT manqingwang hmstenetahierarchicalmultiscaletopologicalenhancednetworkbasedoneegandeogfordrivervigilanceestimation AT guilaizuo hmstenetahierarchicalmultiscaletopologicalenhancednetworkbasedoneegandeogfordrivervigilanceestimation AT dongruigao hmstenetahierarchicalmultiscaletopologicalenhancednetworkbasedoneegandeogfordrivervigilanceestimation |