Trust-Aware Cross-Modal Inconsistency Fusion for Text-Image Fake News Detection in Social Media

Abstract

Multimodal fake-news detection is important because misleading social media posts often combine textual claims with visual content to construct deceptive narratives. This study proposes T-CAMIF, a Trust-Aware Cross-Modal Inconsistency Fusion framework for detecting fake news in paired text-image social media posts. The framework models three complementary evidence sources: textual semantics, visual representations, and cross-modal inconsistency. Textual features were extracted using a Sentence Transformer/MiniLM-based encoder, while visual features were extracted using a MobileNetV2-based encoder. Cross-modal inconsistency was represented using cosine similarity, cosine distance, Euclidean distance, absolute feature difference, and element-wise product. Evidence-level predictions were integrated through a trust-aware fusion gate that incorporated expert probabilities, confidence scores, uncertainty indicators, and disagreement measures. The framework was evaluated on a public Twitter-Weibo multimodal fake-news dataset using accuracy, precision, recall, F1-score, macro-F1, and AUC. Experimental results showed that T-CAMIF achieved the strongest in-domain performance on the combined dataset, with an accuracy of 0.9061, F1-score of 0.9095, macro-F1 of 0.9059, and AUC of 0.9709. It outperformed unimodal, inconsistency-only, simple concatenation, and static full-fusion baselines. Ablation and robustness analyses confirmed the contribution of cross-modal inconsistency and trust-aware fusion, while also showing reduced performance under missing modalities and mismatched image-text pairs. Cross-platform evaluation further indicated that platform variation remains a key challenge. These findings suggest that trust-aware modelling of text-image relationships can improve multimodal fake-news detection and motivate future