Deepfake detection method based on patch-wise lighting inconsistency
The rapid development and widespread dissemination of deepfake techniques has caused increased concern.The malicious application of deepfake techniques also poses a potential threat to the society.Therefore, how to detect deepfake content has become a popular research topic.Most of the previous deep...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
POSTS&TELECOM PRESS Co., LTD
2023-02-01
|
| Series: | 网络与信息安全学报 |
| Subjects: | |
| Online Access: | http://www.cjnis.com.cn/thesisDetails#10.11959/j.issn.2096-109x.2023011 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841529652767621120 |
|---|---|
| author | Wenxuan WU Wenbo ZHOU Weiming ZHANG Nenghai YU |
| author_facet | Wenxuan WU Wenbo ZHOU Weiming ZHANG Nenghai YU |
| author_sort | Wenxuan WU |
| collection | DOAJ |
| description | The rapid development and widespread dissemination of deepfake techniques has caused increased concern.The malicious application of deepfake techniques also poses a potential threat to the society.Therefore, how to detect deepfake content has become a popular research topic.Most of the previous deepfake detection algorithms focused on capturing subtle forgery traces at pixel level and have achieved some results.However, most of the deepfake algorithms ignore the lighting information before and after generation, resulting in some lighting inconsistency between the original face and the forged face, which provided the possibility of using lighting inconsistency to detect deepfake.A corresponding algorithm was designed from two perspectives: introducing lighting inconsistency information and designing a network structure module for a specific task.For the introduction of lighting task, a new network structure was derived by designing the corresponding channel fusion method to provide more lighting inconsistency information to the network feature extraction layer.In order to ensure the portability of the network structure, the process of feature channel fusion was placed before the network extraction information, so that the proposed method can be fully planted to common deepfake detection networks.For the design of the network structure, a deepfake detection method was proposed for lighting inconsistency based on patch-similarity from two perspectives: network structure and loss function design.For the network structure, based on the characteristic of inconsistency between the forged image tampering region and the background region, the extracted features were chunked in the network feature layer and the feature layer similarity matrix was obtained by comparing the patch-wise cosine similarity to make the network focus more on the lighting inconsistency.On this basis, based on the feature layer similarity matching scheme, an independent ground truth and loss function was designed for this task in a targeted manner by comparing the input image with the untampered image of this image for patch-wise authenticity.It is demonstrated experimentally that the accuracy of the proposed method is significantly improved for deepfake detection compared with the baseline method. |
| format | Article |
| id | doaj-art-d90ad7ed34ee4324a15f29ff595f340b |
| institution | Kabale University |
| issn | 2096-109X |
| language | English |
| publishDate | 2023-02-01 |
| publisher | POSTS&TELECOM PRESS Co., LTD |
| record_format | Article |
| series | 网络与信息安全学报 |
| spelling | doaj-art-d90ad7ed34ee4324a15f29ff595f340b2025-01-15T03:16:33ZengPOSTS&TELECOM PRESS Co., LTD网络与信息安全学报2096-109X2023-02-01916717759577722Deepfake detection method based on patch-wise lighting inconsistencyWenxuan WUWenbo ZHOUWeiming ZHANGNenghai YUThe rapid development and widespread dissemination of deepfake techniques has caused increased concern.The malicious application of deepfake techniques also poses a potential threat to the society.Therefore, how to detect deepfake content has become a popular research topic.Most of the previous deepfake detection algorithms focused on capturing subtle forgery traces at pixel level and have achieved some results.However, most of the deepfake algorithms ignore the lighting information before and after generation, resulting in some lighting inconsistency between the original face and the forged face, which provided the possibility of using lighting inconsistency to detect deepfake.A corresponding algorithm was designed from two perspectives: introducing lighting inconsistency information and designing a network structure module for a specific task.For the introduction of lighting task, a new network structure was derived by designing the corresponding channel fusion method to provide more lighting inconsistency information to the network feature extraction layer.In order to ensure the portability of the network structure, the process of feature channel fusion was placed before the network extraction information, so that the proposed method can be fully planted to common deepfake detection networks.For the design of the network structure, a deepfake detection method was proposed for lighting inconsistency based on patch-similarity from two perspectives: network structure and loss function design.For the network structure, based on the characteristic of inconsistency between the forged image tampering region and the background region, the extracted features were chunked in the network feature layer and the feature layer similarity matrix was obtained by comparing the patch-wise cosine similarity to make the network focus more on the lighting inconsistency.On this basis, based on the feature layer similarity matching scheme, an independent ground truth and loss function was designed for this task in a targeted manner by comparing the input image with the untampered image of this image for patch-wise authenticity.It is demonstrated experimentally that the accuracy of the proposed method is significantly improved for deepfake detection compared with the baseline method.http://www.cjnis.com.cn/thesisDetails#10.11959/j.issn.2096-109x.2023011Deepfake detectionlighting inconsistencycolor constancypatch-similarity comparison |
| spellingShingle | Wenxuan WU Wenbo ZHOU Weiming ZHANG Nenghai YU Deepfake detection method based on patch-wise lighting inconsistency 网络与信息安全学报 Deepfake detection lighting inconsistency color constancy patch-similarity comparison |
| title | Deepfake detection method based on patch-wise lighting inconsistency |
| title_full | Deepfake detection method based on patch-wise lighting inconsistency |
| title_fullStr | Deepfake detection method based on patch-wise lighting inconsistency |
| title_full_unstemmed | Deepfake detection method based on patch-wise lighting inconsistency |
| title_short | Deepfake detection method based on patch-wise lighting inconsistency |
| title_sort | deepfake detection method based on patch wise lighting inconsistency |
| topic | Deepfake detection lighting inconsistency color constancy patch-similarity comparison |
| url | http://www.cjnis.com.cn/thesisDetails#10.11959/j.issn.2096-109x.2023011 |
| work_keys_str_mv | AT wenxuanwu deepfakedetectionmethodbasedonpatchwiselightinginconsistency AT wenbozhou deepfakedetectionmethodbasedonpatchwiselightinginconsistency AT weimingzhang deepfakedetectionmethodbasedonpatchwiselightinginconsistency AT nenghaiyu deepfakedetectionmethodbasedonpatchwiselightinginconsistency |