Multimodal Fake News Detection with Contrastive Learning and Optimal Transport
IntroductionThe proliferation of social media platforms has facilitated the spread of fake news, posing significant risks to public perception and societal stability. Existing methods for multimodal fake news detection have made important progress in combining textual and visual information but stil...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2024-11-01
|
| Series: | Frontiers in Computer Science |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fcomp.2024.1473457/full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846164234517348352 |
|---|---|
| author | Xiaorong Shen Xiaorong Shen Maowei Huang Zheng Hu Shimin Cai Tao Zhou |
| author_facet | Xiaorong Shen Xiaorong Shen Maowei Huang Zheng Hu Shimin Cai Tao Zhou |
| author_sort | Xiaorong Shen |
| collection | DOAJ |
| description | IntroductionThe proliferation of social media platforms has facilitated the spread of fake news, posing significant risks to public perception and societal stability. Existing methods for multimodal fake news detection have made important progress in combining textual and visual information but still face challenges in effectively aligning and merging these different types of data. These challenges often result in incomplete or inaccurate feature representations, thereby limiting overall performance.MethodsTo address these limitations, we propose a novel framework named MCOT (Multimodal Fake News Detection with Contrastive Learning and Optimal Transport). MCOT integrates textual and visual information through three key components: cross-modal attention mechanism, contrastive learning, and optimal transport. Specifically, we first use cross-modal attention mechanism to enhance the interaction between text and image features. Then, we employ contrastive learning to align related embeddings while distinguishing unrelated pairs, and we apply optimal transport to refine the alignment of feature distributions across modalities.ResultsThis integrated approach results in more precise and robust feature representations, thus enhancing detection accuracy. Experimental results on two public datasets demonstrate that the proposed MCOT outperforms state-of-the-art methods.DiscussionOur future work will focus on improving its generalization and expanding its capabilities to additional modalities. |
| format | Article |
| id | doaj-art-cdb5b5653dcc4ac89f9bbcfd4ed69b2e |
| institution | Kabale University |
| issn | 2624-9898 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Computer Science |
| spelling | doaj-art-cdb5b5653dcc4ac89f9bbcfd4ed69b2e2024-11-18T12:17:44ZengFrontiers Media S.A.Frontiers in Computer Science2624-98982024-11-01610.3389/fcomp.2024.14734571473457Multimodal Fake News Detection with Contrastive Learning and Optimal TransportXiaorong Shen0Xiaorong Shen1Maowei Huang2Zheng Hu3Shimin Cai4Tao Zhou5Big Data Research Center, University of Electronic Science and Technology of China, Chengdu, Chinai-Large Model Innovation Lab of Ideological and Political Science, University of Electronic Science and Technology of China, Chengdu, ChinaBig Data Research Center, University of Electronic Science and Technology of China, Chengdu, ChinaBig Data Research Center, University of Electronic Science and Technology of China, Chengdu, ChinaBig Data Research Center, University of Electronic Science and Technology of China, Chengdu, ChinaBig Data Research Center, University of Electronic Science and Technology of China, Chengdu, ChinaIntroductionThe proliferation of social media platforms has facilitated the spread of fake news, posing significant risks to public perception and societal stability. Existing methods for multimodal fake news detection have made important progress in combining textual and visual information but still face challenges in effectively aligning and merging these different types of data. These challenges often result in incomplete or inaccurate feature representations, thereby limiting overall performance.MethodsTo address these limitations, we propose a novel framework named MCOT (Multimodal Fake News Detection with Contrastive Learning and Optimal Transport). MCOT integrates textual and visual information through three key components: cross-modal attention mechanism, contrastive learning, and optimal transport. Specifically, we first use cross-modal attention mechanism to enhance the interaction between text and image features. Then, we employ contrastive learning to align related embeddings while distinguishing unrelated pairs, and we apply optimal transport to refine the alignment of feature distributions across modalities.ResultsThis integrated approach results in more precise and robust feature representations, thus enhancing detection accuracy. Experimental results on two public datasets demonstrate that the proposed MCOT outperforms state-of-the-art methods.DiscussionOur future work will focus on improving its generalization and expanding its capabilities to additional modalities.https://www.frontiersin.org/articles/10.3389/fcomp.2024.1473457/fullfake news detectionmultimodal datacross-modal attentioncontrastive learningoptimal transport |
| spellingShingle | Xiaorong Shen Xiaorong Shen Maowei Huang Zheng Hu Shimin Cai Tao Zhou Multimodal Fake News Detection with Contrastive Learning and Optimal Transport Frontiers in Computer Science fake news detection multimodal data cross-modal attention contrastive learning optimal transport |
| title | Multimodal Fake News Detection with Contrastive Learning and Optimal Transport |
| title_full | Multimodal Fake News Detection with Contrastive Learning and Optimal Transport |
| title_fullStr | Multimodal Fake News Detection with Contrastive Learning and Optimal Transport |
| title_full_unstemmed | Multimodal Fake News Detection with Contrastive Learning and Optimal Transport |
| title_short | Multimodal Fake News Detection with Contrastive Learning and Optimal Transport |
| title_sort | multimodal fake news detection with contrastive learning and optimal transport |
| topic | fake news detection multimodal data cross-modal attention contrastive learning optimal transport |
| url | https://www.frontiersin.org/articles/10.3389/fcomp.2024.1473457/full |
| work_keys_str_mv | AT xiaorongshen multimodalfakenewsdetectionwithcontrastivelearningandoptimaltransport AT xiaorongshen multimodalfakenewsdetectionwithcontrastivelearningandoptimaltransport AT maoweihuang multimodalfakenewsdetectionwithcontrastivelearningandoptimaltransport AT zhenghu multimodalfakenewsdetectionwithcontrastivelearningandoptimaltransport AT shimincai multimodalfakenewsdetectionwithcontrastivelearningandoptimaltransport AT taozhou multimodalfakenewsdetectionwithcontrastivelearningandoptimaltransport |