Evaluation framework for deepfake speech detection: a comparative study of state-of-the-art deepfake speech detectors
Abstract The proliferation of deepfake speech poses a significant threat to cybersecurity, from manipulating political speeches and impersonating public figures to spoofing voice biometric systems. The increasing sophistication of adversaries increases the necessity of deploying adaptive detection m...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2025-08-01
|
| Series: | Cybersecurity |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s42400-024-00346-1 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849342169465225216 |
|---|---|
| author | Anton Firc Kamil Malinka Petr Hanáček |
| author_facet | Anton Firc Kamil Malinka Petr Hanáček |
| author_sort | Anton Firc |
| collection | DOAJ |
| description | Abstract The proliferation of deepfake speech poses a significant threat to cybersecurity, from manipulating political speeches and impersonating public figures to spoofing voice biometric systems. The increasing sophistication of adversaries increases the necessity of deploying adaptive detection methods. Moreover, real-world incidents such as fraudulent financial transactions highlight the severity of the problem. Although numerous detectors have been developed, their evaluation remains difficult due to different methodologies and benchmark datasets, making direct comparisons impossible. This study presents a general and detailed framework for evaluating and comparing deepfake speech detectors. We further demonstrate the use of this framework to evaluate 40 state-of-the-art deepfake speech detectors under various conditions and data samples. We objectively compare these methods and identify the key attributes influencing performance the most. We also stress the issue of generalisation, as current detectors struggle to detect previously unseen deepfake speech samples or samples that have been modified. Finally, to strengthen the defence against synthetic audio content, we provide recommendations for improving the robustness of future detectors. |
| format | Article |
| id | doaj-art-dbf5a4e115fe488e8d5dfbbf45e2a76a |
| institution | Kabale University |
| issn | 2523-3246 |
| language | English |
| publishDate | 2025-08-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | Cybersecurity |
| spelling | doaj-art-dbf5a4e115fe488e8d5dfbbf45e2a76a2025-08-20T03:43:27ZengSpringerOpenCybersecurity2523-32462025-08-018112410.1186/s42400-024-00346-1Evaluation framework for deepfake speech detection: a comparative study of state-of-the-art deepfake speech detectorsAnton Firc0Kamil Malinka1Petr Hanáček2Faculty of Information Technology, Brno University of TechnologyFaculty of Information Technology, Brno University of TechnologyFaculty of Information Technology, Brno University of TechnologyAbstract The proliferation of deepfake speech poses a significant threat to cybersecurity, from manipulating political speeches and impersonating public figures to spoofing voice biometric systems. The increasing sophistication of adversaries increases the necessity of deploying adaptive detection methods. Moreover, real-world incidents such as fraudulent financial transactions highlight the severity of the problem. Although numerous detectors have been developed, their evaluation remains difficult due to different methodologies and benchmark datasets, making direct comparisons impossible. This study presents a general and detailed framework for evaluating and comparing deepfake speech detectors. We further demonstrate the use of this framework to evaluate 40 state-of-the-art deepfake speech detectors under various conditions and data samples. We objectively compare these methods and identify the key attributes influencing performance the most. We also stress the issue of generalisation, as current detectors struggle to detect previously unseen deepfake speech samples or samples that have been modified. Finally, to strengthen the defence against synthetic audio content, we provide recommendations for improving the robustness of future detectors.https://doi.org/10.1186/s42400-024-00346-1Deepfake speechDetectionRobustnessEvaluation frameworkComputer security |
| spellingShingle | Anton Firc Kamil Malinka Petr Hanáček Evaluation framework for deepfake speech detection: a comparative study of state-of-the-art deepfake speech detectors Cybersecurity Deepfake speech Detection Robustness Evaluation framework Computer security |
| title | Evaluation framework for deepfake speech detection: a comparative study of state-of-the-art deepfake speech detectors |
| title_full | Evaluation framework for deepfake speech detection: a comparative study of state-of-the-art deepfake speech detectors |
| title_fullStr | Evaluation framework for deepfake speech detection: a comparative study of state-of-the-art deepfake speech detectors |
| title_full_unstemmed | Evaluation framework for deepfake speech detection: a comparative study of state-of-the-art deepfake speech detectors |
| title_short | Evaluation framework for deepfake speech detection: a comparative study of state-of-the-art deepfake speech detectors |
| title_sort | evaluation framework for deepfake speech detection a comparative study of state of the art deepfake speech detectors |
| topic | Deepfake speech Detection Robustness Evaluation framework Computer security |
| url | https://doi.org/10.1186/s42400-024-00346-1 |
| work_keys_str_mv | AT antonfirc evaluationframeworkfordeepfakespeechdetectionacomparativestudyofstateoftheartdeepfakespeechdetectors AT kamilmalinka evaluationframeworkfordeepfakespeechdetectionacomparativestudyofstateoftheartdeepfakespeechdetectors AT petrhanacek evaluationframeworkfordeepfakespeechdetectionacomparativestudyofstateoftheartdeepfakespeechdetectors |