Multimodal consumer choice prediction using EEG signals and eye tracking
Marketing plays a vital role in the success of a business, driving customer engagement, brand recognition, and revenue growth. Neuromarketing adds depth to this by employing insights into consumer behavior through brain activity and emotional responses to create more effective marketing strategies....
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2025-01-01
|
| Series: | Frontiers in Computational Neuroscience |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fncom.2024.1516440/full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841555068222963712 |
|---|---|
| author | Syed Muhammad Usman Shehzad Khalid Aimen Tanveer Ali Shariq Imran Muhammad Zubair |
| author_facet | Syed Muhammad Usman Shehzad Khalid Aimen Tanveer Ali Shariq Imran Muhammad Zubair |
| author_sort | Syed Muhammad Usman |
| collection | DOAJ |
| description | Marketing plays a vital role in the success of a business, driving customer engagement, brand recognition, and revenue growth. Neuromarketing adds depth to this by employing insights into consumer behavior through brain activity and emotional responses to create more effective marketing strategies. Electroencephalogram (EEG) has typically been utilized by researchers for neuromarketing, whereas Eye Tracking (ET) has remained unexplored. To address this gap, we propose a novel multimodal approach to predict consumer choices by integrating EEG and ET data. Noise from EEG signals is mitigated using a bandpass filter, Artifact Subspace Reconstruction (ASR), and Fast Orthogonal Regression for Classification and Estimation (FORCE). Class imbalance is handled by employing the Synthetic Minority Over-sampling Technique (SMOTE). Handcrafted features, including statistical and wavelet features, and automated features from Convolutional Neural Network and Long Short-Term Memory (CNN-LSTM), have been extracted and concatenated to generate a feature space representation. For ET data, preprocessing involved interpolation, gaze plots, and SMOTE, followed by feature extraction using LeNet-5 and handcrafted features like fixations and saccades. Multimodal feature space representation was generated by performing feature-level fusion for EEG and ET, which was later fed into a meta-learner-based ensemble classifier with three base classifiers, including Random Forest, Extended Gradient Boosting, and Gradient Boosting, and Random Forest as the meta-classifier, to perform classification between buy vs. not buy. The performance of the proposed approach is evaluated using a variety of performance metrics, including accuracy, precision, recall, and F1 score. Our model demonstrated superior performance compared to competitors by achieving 84.01% accuracy in predicting consumer choices and 83% precision in identifying positive consumer preferences. |
| format | Article |
| id | doaj-art-26a63cf21419403fb62550857cad5458 |
| institution | Kabale University |
| issn | 1662-5188 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Computational Neuroscience |
| spelling | doaj-art-26a63cf21419403fb62550857cad54582025-01-08T06:12:15ZengFrontiers Media S.A.Frontiers in Computational Neuroscience1662-51882025-01-011810.3389/fncom.2024.15164401516440Multimodal consumer choice prediction using EEG signals and eye trackingSyed Muhammad Usman0Shehzad Khalid1Aimen Tanveer2Ali Shariq Imran3Muhammad Zubair4Department of Computer Science, Bahria School of Engineering and Applied Science, Bahria University, Islamabad, PakistanDepartment of Computer Engineering, Bahria School of Engineering and Applied Science, Bahria University, Islamabad, PakistanDepartment of Creative Technologies, Air University, Islamabad, PakistanDepartment of Computer Science, Norwegian University of Science and Technology, Gjøvik, NorwayInterdisciplinary Research Center for Finance and Digital Economy, King Fahd University of Petroleum and Minerals, Dhahran, Saudi ArabiaMarketing plays a vital role in the success of a business, driving customer engagement, brand recognition, and revenue growth. Neuromarketing adds depth to this by employing insights into consumer behavior through brain activity and emotional responses to create more effective marketing strategies. Electroencephalogram (EEG) has typically been utilized by researchers for neuromarketing, whereas Eye Tracking (ET) has remained unexplored. To address this gap, we propose a novel multimodal approach to predict consumer choices by integrating EEG and ET data. Noise from EEG signals is mitigated using a bandpass filter, Artifact Subspace Reconstruction (ASR), and Fast Orthogonal Regression for Classification and Estimation (FORCE). Class imbalance is handled by employing the Synthetic Minority Over-sampling Technique (SMOTE). Handcrafted features, including statistical and wavelet features, and automated features from Convolutional Neural Network and Long Short-Term Memory (CNN-LSTM), have been extracted and concatenated to generate a feature space representation. For ET data, preprocessing involved interpolation, gaze plots, and SMOTE, followed by feature extraction using LeNet-5 and handcrafted features like fixations and saccades. Multimodal feature space representation was generated by performing feature-level fusion for EEG and ET, which was later fed into a meta-learner-based ensemble classifier with three base classifiers, including Random Forest, Extended Gradient Boosting, and Gradient Boosting, and Random Forest as the meta-classifier, to perform classification between buy vs. not buy. The performance of the proposed approach is evaluated using a variety of performance metrics, including accuracy, precision, recall, and F1 score. Our model demonstrated superior performance compared to competitors by achieving 84.01% accuracy in predicting consumer choices and 83% precision in identifying positive consumer preferences.https://www.frontiersin.org/articles/10.3389/fncom.2024.1516440/fullEEGeye trackingneuromarketingCNN-LSTMmultimodal |
| spellingShingle | Syed Muhammad Usman Shehzad Khalid Aimen Tanveer Ali Shariq Imran Muhammad Zubair Multimodal consumer choice prediction using EEG signals and eye tracking Frontiers in Computational Neuroscience EEG eye tracking neuromarketing CNN-LSTM multimodal |
| title | Multimodal consumer choice prediction using EEG signals and eye tracking |
| title_full | Multimodal consumer choice prediction using EEG signals and eye tracking |
| title_fullStr | Multimodal consumer choice prediction using EEG signals and eye tracking |
| title_full_unstemmed | Multimodal consumer choice prediction using EEG signals and eye tracking |
| title_short | Multimodal consumer choice prediction using EEG signals and eye tracking |
| title_sort | multimodal consumer choice prediction using eeg signals and eye tracking |
| topic | EEG eye tracking neuromarketing CNN-LSTM multimodal |
| url | https://www.frontiersin.org/articles/10.3389/fncom.2024.1516440/full |
| work_keys_str_mv | AT syedmuhammadusman multimodalconsumerchoicepredictionusingeegsignalsandeyetracking AT shehzadkhalid multimodalconsumerchoicepredictionusingeegsignalsandeyetracking AT aimentanveer multimodalconsumerchoicepredictionusingeegsignalsandeyetracking AT alishariqimran multimodalconsumerchoicepredictionusingeegsignalsandeyetracking AT muhammadzubair multimodalconsumerchoicepredictionusingeegsignalsandeyetracking |