Driving-Related Cognitive Abilities Prediction Based on Transformer’s Multimodal Fusion Framework
With the increasing complexity of urban roads and rising traffic flow, traffic safety has become a critical societal concern. Current research primarily addresses drivers’ attention, reaction speed, and perceptual abilities, but comprehensive assessments of cognitive abilities in complex traffic env...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-12-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/25/1/174 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841548946749521920 |
|---|---|
| author | Yifan Li Bo Liu Wenli Zhang |
| author_facet | Yifan Li Bo Liu Wenli Zhang |
| author_sort | Yifan Li |
| collection | DOAJ |
| description | With the increasing complexity of urban roads and rising traffic flow, traffic safety has become a critical societal concern. Current research primarily addresses drivers’ attention, reaction speed, and perceptual abilities, but comprehensive assessments of cognitive abilities in complex traffic environments are lacking. This study, grounded in cognitive science and neuropsychology, identifies and quantitatively evaluates ten cognitive components related to driving decision-making, execution, and psychological states by analyzing video footage of drivers’ actions. Physiological data (e.g., Electrocardiogram (ECG), Electrodermal Activity (EDA)) and non-physiological data (e.g., Eye Tracking (ET)) are collected from simulated driving scenarios. A dual-branch Transformer network model is developed to extract temporal features from multimodal data, integrating these features through a weight adjustment strategy to predict driving-related cognitive abilities. Experiments on a multimodal driving dataset from the Computational Physiology Laboratory at the University of Houston, USA, yield an Accuracy (ACC) of 0.9908 and an F1-score of 0.9832, confirming the model’s effectiveness. This method effectively combines scale measurements and driving behavior under secondary tasks to assess cognitive abilities, providing a novel approach for driving risk assessment and traffic safety strategy development. |
| format | Article |
| id | doaj-art-c1fbcc48a3184bf9a36c52df78426ce5 |
| institution | Kabale University |
| issn | 1424-8220 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj-art-c1fbcc48a3184bf9a36c52df78426ce52025-01-10T13:21:07ZengMDPI AGSensors1424-82202024-12-0125117410.3390/s25010174Driving-Related Cognitive Abilities Prediction Based on Transformer’s Multimodal Fusion FrameworkYifan Li0Bo Liu1Wenli Zhang2Faculty of Information Science and Technology, Beijing University of Technology, Beijing 100124, ChinaFaculty of Information Science and Technology, Beijing University of Technology, Beijing 100124, ChinaFaculty of Information Science and Technology, Beijing University of Technology, Beijing 100124, ChinaWith the increasing complexity of urban roads and rising traffic flow, traffic safety has become a critical societal concern. Current research primarily addresses drivers’ attention, reaction speed, and perceptual abilities, but comprehensive assessments of cognitive abilities in complex traffic environments are lacking. This study, grounded in cognitive science and neuropsychology, identifies and quantitatively evaluates ten cognitive components related to driving decision-making, execution, and psychological states by analyzing video footage of drivers’ actions. Physiological data (e.g., Electrocardiogram (ECG), Electrodermal Activity (EDA)) and non-physiological data (e.g., Eye Tracking (ET)) are collected from simulated driving scenarios. A dual-branch Transformer network model is developed to extract temporal features from multimodal data, integrating these features through a weight adjustment strategy to predict driving-related cognitive abilities. Experiments on a multimodal driving dataset from the Computational Physiology Laboratory at the University of Houston, USA, yield an Accuracy (ACC) of 0.9908 and an F1-score of 0.9832, confirming the model’s effectiveness. This method effectively combines scale measurements and driving behavior under secondary tasks to assess cognitive abilities, providing a novel approach for driving risk assessment and traffic safety strategy development.https://www.mdpi.com/1424-8220/25/1/174biosignalsdriving-related cognitive abilitiesmultimodaldriving safety |
| spellingShingle | Yifan Li Bo Liu Wenli Zhang Driving-Related Cognitive Abilities Prediction Based on Transformer’s Multimodal Fusion Framework Sensors biosignals driving-related cognitive abilities multimodal driving safety |
| title | Driving-Related Cognitive Abilities Prediction Based on Transformer’s Multimodal Fusion Framework |
| title_full | Driving-Related Cognitive Abilities Prediction Based on Transformer’s Multimodal Fusion Framework |
| title_fullStr | Driving-Related Cognitive Abilities Prediction Based on Transformer’s Multimodal Fusion Framework |
| title_full_unstemmed | Driving-Related Cognitive Abilities Prediction Based on Transformer’s Multimodal Fusion Framework |
| title_short | Driving-Related Cognitive Abilities Prediction Based on Transformer’s Multimodal Fusion Framework |
| title_sort | driving related cognitive abilities prediction based on transformer s multimodal fusion framework |
| topic | biosignals driving-related cognitive abilities multimodal driving safety |
| url | https://www.mdpi.com/1424-8220/25/1/174 |
| work_keys_str_mv | AT yifanli drivingrelatedcognitiveabilitiespredictionbasedontransformersmultimodalfusionframework AT boliu drivingrelatedcognitiveabilitiespredictionbasedontransformersmultimodalfusionframework AT wenlizhang drivingrelatedcognitiveabilitiespredictionbasedontransformersmultimodalfusionframework |