Feature-level fusion of face and speech based multimodal biometric attendance system with liveness detection
The rapid growth of deep learning and the Internet of Things has spurred the need for touchless biometric systems in areas where cleanliness and non-intrusive user interaction are critical. In order to authenticate seamlessly, traditional biometric methods such as fingerprint, hand, etc. recognition...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
AIP Publishing LLC
2024-11-01
|
| Series: | AIP Advances |
| Online Access: | http://dx.doi.org/10.1063/5.0234430 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846141168511877120 |
|---|---|
| author | Khushboo Jha Aruna Jain Sumit Srivastava |
| author_facet | Khushboo Jha Aruna Jain Sumit Srivastava |
| author_sort | Khushboo Jha |
| collection | DOAJ |
| description | The rapid growth of deep learning and the Internet of Things has spurred the need for touchless biometric systems in areas where cleanliness and non-intrusive user interaction are critical. In order to authenticate seamlessly, traditional biometric methods such as fingerprint, hand, etc. recognition require physical contact and, therefore, risk hygiene issues, hence making face and speaker verification more viable alternatives. A robust Multimodal Biometric Attendance System (MBAS) is needed due to the vulnerabilities and limitations of single modality systems. In this research, we introduce MBAS using feature-level fusion of speech data with face data, combining the best of both worlds. The textural features based on a person’s facial appearance are integrated with dynamic speech information for liveness detection, followed by dimensionality reduction using linear discriminant analysis, and then incorporated into a Bi-LSTM classifier. Therefore, for better security, accuracy, and anti-spoofing attacks, this approach is proposed in addition to increasing accuracy as well as enhancing security against spoofing attacks. Two publicly available datasets, DeepfakeTIMIT and AVSpeech, are extensively explored to evaluate different fusion strategies, classifier types, and standard performance metrics. The proposed system outperformed other cutting-edge biometric based systems by exhibiting a 97.51% high accuracy rate with a precision of 99.10% and an equal error rate of 2.48%. These findings affirm the effectiveness and possible real-world applications of the MBAS concept, along with its enhancement ensuring safety. Furthermore, this study underscores the importance of incorporating advanced liveness detection into secure contactless biometrics solutions for modern attendance management in various industries that encompass both face and voice modalities. |
| format | Article |
| id | doaj-art-16cc963f22ad4e9a81e75e64cb455c46 |
| institution | Kabale University |
| issn | 2158-3226 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | AIP Publishing LLC |
| record_format | Article |
| series | AIP Advances |
| spelling | doaj-art-16cc963f22ad4e9a81e75e64cb455c462024-12-04T16:59:16ZengAIP Publishing LLCAIP Advances2158-32262024-11-011411115007115007-1010.1063/5.0234430Feature-level fusion of face and speech based multimodal biometric attendance system with liveness detectionKhushboo Jha0Aruna Jain1Sumit Srivastava2Department of Computer Science and Engineering, Birla Institute of Technology, Ranchi 835215, IndiaDepartment of Computer Science and Engineering, Birla Institute of Technology, Ranchi 835215, IndiaDepartment of Computer Science and Engineering, Birla Institute of Technology, Ranchi 835215, IndiaThe rapid growth of deep learning and the Internet of Things has spurred the need for touchless biometric systems in areas where cleanliness and non-intrusive user interaction are critical. In order to authenticate seamlessly, traditional biometric methods such as fingerprint, hand, etc. recognition require physical contact and, therefore, risk hygiene issues, hence making face and speaker verification more viable alternatives. A robust Multimodal Biometric Attendance System (MBAS) is needed due to the vulnerabilities and limitations of single modality systems. In this research, we introduce MBAS using feature-level fusion of speech data with face data, combining the best of both worlds. The textural features based on a person’s facial appearance are integrated with dynamic speech information for liveness detection, followed by dimensionality reduction using linear discriminant analysis, and then incorporated into a Bi-LSTM classifier. Therefore, for better security, accuracy, and anti-spoofing attacks, this approach is proposed in addition to increasing accuracy as well as enhancing security against spoofing attacks. Two publicly available datasets, DeepfakeTIMIT and AVSpeech, are extensively explored to evaluate different fusion strategies, classifier types, and standard performance metrics. The proposed system outperformed other cutting-edge biometric based systems by exhibiting a 97.51% high accuracy rate with a precision of 99.10% and an equal error rate of 2.48%. These findings affirm the effectiveness and possible real-world applications of the MBAS concept, along with its enhancement ensuring safety. Furthermore, this study underscores the importance of incorporating advanced liveness detection into secure contactless biometrics solutions for modern attendance management in various industries that encompass both face and voice modalities.http://dx.doi.org/10.1063/5.0234430 |
| spellingShingle | Khushboo Jha Aruna Jain Sumit Srivastava Feature-level fusion of face and speech based multimodal biometric attendance system with liveness detection AIP Advances |
| title | Feature-level fusion of face and speech based multimodal biometric attendance system with liveness detection |
| title_full | Feature-level fusion of face and speech based multimodal biometric attendance system with liveness detection |
| title_fullStr | Feature-level fusion of face and speech based multimodal biometric attendance system with liveness detection |
| title_full_unstemmed | Feature-level fusion of face and speech based multimodal biometric attendance system with liveness detection |
| title_short | Feature-level fusion of face and speech based multimodal biometric attendance system with liveness detection |
| title_sort | feature level fusion of face and speech based multimodal biometric attendance system with liveness detection |
| url | http://dx.doi.org/10.1063/5.0234430 |
| work_keys_str_mv | AT khushboojha featurelevelfusionoffaceandspeechbasedmultimodalbiometricattendancesystemwithlivenessdetection AT arunajain featurelevelfusionoffaceandspeechbasedmultimodalbiometricattendancesystemwithlivenessdetection AT sumitsrivastava featurelevelfusionoffaceandspeechbasedmultimodalbiometricattendancesystemwithlivenessdetection |