Leveraging Intra and Inter Modality Relationship for Multimodal Fake News Detection

Citations Over TimeTop 1% of 2022 papers

Abstract

Recent years have witnessed a massive growth in the proliferation of fake news online. User-generated content is a blend of text and visual information leading to producing different variants of fake news. As a result, researchers started targeting multimodal methods for fake news detection. Existing methods capture high-level information from different modalities and jointly model them to decide. Given multiple input modalities, we hypothesize that not all modalities may be equally responsible for decision-making. Hence, this paper presents a novel architecture that effectively identifies and suppresses information from weaker modalities and extracts relevant information from the strong modality on a per-sample basis. We also establish intra-modality relationship by extracting fine-grained image and text features. We conduct extensive experiments on real-world datasets to show that our approach outperforms the state-of-the-art by an average of 3.05% and 4.525% on accuracy and F1-score, respectively. We also release the code, implementation details, and model checkpoints for the community’s interest.1

Related Papers

• → Leveraging Intra and Inter Modality Relationship for Multimodal Fake News Detection(2022)69 cited
• → Factors Influencing Modality Choice in Multimodal Applications(2008)15 cited
• The Mismatch of Modalities and Its Effects(2012)
• → Study of communication modalities for teaching distance information(2022)
• → TYPES OF MODALITY AND ITS INCONSISTENCIES/ԵՂԱՆԱԿԱՎՈՐՄԱՆ ՏԵՍԱԿՆԵՐԸ ԵՎ ԴՐԱ ՀԵՏ ԿԱՊՎԱԾ ԱՆՀԱՄԱՊԱՏԱՍԽԱՆՈՒԹՅՈՒՆՆԵՐԸ/ТИПЫ МОДАЛЬНОСТИ И ЕЕ НЕСООТВЕТСТВИЯ(2022)