FBI-Net: Frequency-Based Image Forgery Localization via Multitask Learning With Self-Attention
Image forgery is easily manufactured for illegal acts such as spreading misleading information, which can have unfortunate consequences for society. In this work, we propose a Discrete Cosine Transformation (DCT) based multi-task learning network named FBI-Net, for forgery localization. Our proposed...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9793665/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841536179095207936 |
|---|---|
| author | A-Rom Gu Ju-Hyeon Nam Sang-Chul Lee |
| author_facet | A-Rom Gu Ju-Hyeon Nam Sang-Chul Lee |
| author_sort | A-Rom Gu |
| collection | DOAJ |
| description | Image forgery is easily manufactured for illegal acts such as spreading misleading information, which can have unfortunate consequences for society. In this work, we propose a Discrete Cosine Transformation (DCT) based multi-task learning network named FBI-Net, for forgery localization. Our proposed network adopts a fully convolutional encoder-decoder architecture, consisting of three encoders sharing parameters, a bridge attention module, and two output streams in the decoder. The encoder takes three inputs: RGB images and high-/low- DCT-filtered images. High-frequency components help learn object characteristics that improve CNN accuracy; low-frequency components are essential frequency information to keep most of the energy found in the typical DCT. Subsequently, Dilated Frequency Self-Attention Module, DFSAM in the bridge layer, is incorporated into the network to recalibrate the fused features and enhance the representation. Finally, in the decoder stage, region and edge information of the label are learned through multi-task learning to provide more extensive supervision for forged region localization; the edge stream will give a deeper understanding of features between forged and authentic images and help learn how to predict exquisite representations in images. Simultaneously, the auxiliary features from the pre-trained segmentation model are fused to separate the segmented background and objects, drawing the segmentation result of the dense region obtained. Extensive experiments show that our proposed FBI-Net outperforms existing forgery localization methods on six benchmark splicing and copy-move image datasets, CASIA TIDE v 1.0, CASIA TIDE v 2.0, Carvalho, Columbia, Coverage, IMD2020, achieving the best performance in an average of IoU of 70.99% and F1-score of 76.98% which is 9.79%, 9.82% higher than the previous method, respectively. |
| format | Article |
| id | doaj-art-6842ad7f94ad426ca50bdcc7c7a1fc18 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-6842ad7f94ad426ca50bdcc7c7a1fc182025-01-15T00:01:15ZengIEEEIEEE Access2169-35362022-01-0110627516276210.1109/ACCESS.2022.31820249793665FBI-Net: Frequency-Based Image Forgery Localization via Multitask Learning With Self-AttentionA-Rom Gu0Ju-Hyeon Nam1Sang-Chul Lee2https://orcid.org/0000-0002-6973-2416Department of Computer Science and Engineering, Inha University, Incheon, South KoreaDepartment of Computer Science and Engineering, Inha University, Incheon, South KoreaDepartment of Computer Science and Engineering, Inha University, Incheon, South KoreaImage forgery is easily manufactured for illegal acts such as spreading misleading information, which can have unfortunate consequences for society. In this work, we propose a Discrete Cosine Transformation (DCT) based multi-task learning network named FBI-Net, for forgery localization. Our proposed network adopts a fully convolutional encoder-decoder architecture, consisting of three encoders sharing parameters, a bridge attention module, and two output streams in the decoder. The encoder takes three inputs: RGB images and high-/low- DCT-filtered images. High-frequency components help learn object characteristics that improve CNN accuracy; low-frequency components are essential frequency information to keep most of the energy found in the typical DCT. Subsequently, Dilated Frequency Self-Attention Module, DFSAM in the bridge layer, is incorporated into the network to recalibrate the fused features and enhance the representation. Finally, in the decoder stage, region and edge information of the label are learned through multi-task learning to provide more extensive supervision for forged region localization; the edge stream will give a deeper understanding of features between forged and authentic images and help learn how to predict exquisite representations in images. Simultaneously, the auxiliary features from the pre-trained segmentation model are fused to separate the segmented background and objects, drawing the segmentation result of the dense region obtained. Extensive experiments show that our proposed FBI-Net outperforms existing forgery localization methods on six benchmark splicing and copy-move image datasets, CASIA TIDE v 1.0, CASIA TIDE v 2.0, Carvalho, Columbia, Coverage, IMD2020, achieving the best performance in an average of IoU of 70.99% and F1-score of 76.98% which is 9.79%, 9.82% higher than the previous method, respectively.https://ieeexplore.ieee.org/document/9793665/Deep learningforgery detectionfrequency domainself-attentionlocalization |
| spellingShingle | A-Rom Gu Ju-Hyeon Nam Sang-Chul Lee FBI-Net: Frequency-Based Image Forgery Localization via Multitask Learning With Self-Attention IEEE Access Deep learning forgery detection frequency domain self-attention localization |
| title | FBI-Net: Frequency-Based Image Forgery Localization via Multitask Learning With Self-Attention |
| title_full | FBI-Net: Frequency-Based Image Forgery Localization via Multitask Learning With Self-Attention |
| title_fullStr | FBI-Net: Frequency-Based Image Forgery Localization via Multitask Learning With Self-Attention |
| title_full_unstemmed | FBI-Net: Frequency-Based Image Forgery Localization via Multitask Learning With Self-Attention |
| title_short | FBI-Net: Frequency-Based Image Forgery Localization via Multitask Learning With Self-Attention |
| title_sort | fbi net frequency based image forgery localization via multitask learning with self attention |
| topic | Deep learning forgery detection frequency domain self-attention localization |
| url | https://ieeexplore.ieee.org/document/9793665/ |
| work_keys_str_mv | AT aromgu fbinetfrequencybasedimageforgerylocalizationviamultitasklearningwithselfattention AT juhyeonnam fbinetfrequencybasedimageforgerylocalizationviamultitasklearningwithselfattention AT sangchullee fbinetfrequencybasedimageforgerylocalizationviamultitasklearningwithselfattention |