Facial and Neck Region Analysis for Deepfake Detection Using Remote Photoplethysmography Signal Similarity
Deepfake (DF) involves utilizing artificial intelligence (AI) technology to synthesize or manipulate images, voices, and other human or object data. However, recent times have seen a surge in instances of DF technology misuse, raising concerns about cybercrime and the credibility of manipulated info...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2024-01-01
|
| Series: | IET Biometrics |
| Online Access: | http://dx.doi.org/10.1049/bme2/7095412 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846150255229272064 |
|---|---|
| author | Byeong Seon An Hyeji Lim Hyeon Ah Seong Eui Chul Lee |
| author_facet | Byeong Seon An Hyeji Lim Hyeon Ah Seong Eui Chul Lee |
| author_sort | Byeong Seon An |
| collection | DOAJ |
| description | Deepfake (DF) involves utilizing artificial intelligence (AI) technology to synthesize or manipulate images, voices, and other human or object data. However, recent times have seen a surge in instances of DF technology misuse, raising concerns about cybercrime and the credibility of manipulated information. The objective of this study is to devise a method that employs remote photoplethysmography (rPPG) biosignals for DF detection. The face was divided into five regions based on landmarks, with automatic extraction performed on the neck region. We conducted rPPG signal extraction from each facial area and the neck region was defined as the ground truth. The five signals extracted from the face were used as inputs to an support vector machine (SVM) model by calculating the euclidean distance between each signal and the signal extracted from the neck region, measuring rPPG signal similarity with five features. Our approach demonstrated robust performance with an area under the curve (AUC) score of 91.2% on the audio-driven dataset and 99.7% on the face swapping generative adversarial network (FSGAN) dataset, even though we only used datasets excluding DF techniques that can be visually identified in Korean DF Detection Dataset (KoDF). Therefore, our research findings demonstrate that similarity features of rPPG signals can be utilized as key features for detecting DFs. |
| format | Article |
| id | doaj-art-89dbf69d3c8c4aa9860c7493dc9b3055 |
| institution | Kabale University |
| issn | 2047-4946 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | Wiley |
| record_format | Article |
| series | IET Biometrics |
| spelling | doaj-art-89dbf69d3c8c4aa9860c7493dc9b30552024-11-29T05:00:03ZengWileyIET Biometrics2047-49462024-01-01202410.1049/bme2/7095412Facial and Neck Region Analysis for Deepfake Detection Using Remote Photoplethysmography Signal SimilarityByeong Seon An0Hyeji Lim1Hyeon Ah Seong2Eui Chul Lee3Department of AI and InformaticsDepartment of AI and InformaticsDepartment of AI and InformaticsDepartment of Human-Centered Artificial IntelligenceDeepfake (DF) involves utilizing artificial intelligence (AI) technology to synthesize or manipulate images, voices, and other human or object data. However, recent times have seen a surge in instances of DF technology misuse, raising concerns about cybercrime and the credibility of manipulated information. The objective of this study is to devise a method that employs remote photoplethysmography (rPPG) biosignals for DF detection. The face was divided into five regions based on landmarks, with automatic extraction performed on the neck region. We conducted rPPG signal extraction from each facial area and the neck region was defined as the ground truth. The five signals extracted from the face were used as inputs to an support vector machine (SVM) model by calculating the euclidean distance between each signal and the signal extracted from the neck region, measuring rPPG signal similarity with five features. Our approach demonstrated robust performance with an area under the curve (AUC) score of 91.2% on the audio-driven dataset and 99.7% on the face swapping generative adversarial network (FSGAN) dataset, even though we only used datasets excluding DF techniques that can be visually identified in Korean DF Detection Dataset (KoDF). Therefore, our research findings demonstrate that similarity features of rPPG signals can be utilized as key features for detecting DFs.http://dx.doi.org/10.1049/bme2/7095412 |
| spellingShingle | Byeong Seon An Hyeji Lim Hyeon Ah Seong Eui Chul Lee Facial and Neck Region Analysis for Deepfake Detection Using Remote Photoplethysmography Signal Similarity IET Biometrics |
| title | Facial and Neck Region Analysis for Deepfake Detection Using Remote Photoplethysmography Signal Similarity |
| title_full | Facial and Neck Region Analysis for Deepfake Detection Using Remote Photoplethysmography Signal Similarity |
| title_fullStr | Facial and Neck Region Analysis for Deepfake Detection Using Remote Photoplethysmography Signal Similarity |
| title_full_unstemmed | Facial and Neck Region Analysis for Deepfake Detection Using Remote Photoplethysmography Signal Similarity |
| title_short | Facial and Neck Region Analysis for Deepfake Detection Using Remote Photoplethysmography Signal Similarity |
| title_sort | facial and neck region analysis for deepfake detection using remote photoplethysmography signal similarity |
| url | http://dx.doi.org/10.1049/bme2/7095412 |
| work_keys_str_mv | AT byeongseonan facialandneckregionanalysisfordeepfakedetectionusingremotephotoplethysmographysignalsimilarity AT hyejilim facialandneckregionanalysisfordeepfakedetectionusingremotephotoplethysmographysignalsimilarity AT hyeonahseong facialandneckregionanalysisfordeepfakedetectionusingremotephotoplethysmographysignalsimilarity AT euichullee facialandneckregionanalysisfordeepfakedetectionusingremotephotoplethysmographysignalsimilarity |