SEMTEX++: Semantic & Threat-aware Explainable Framework for Malicious Content

Abstract

The information that is circulating in social networks is no longer only text but mostly text along with images and videos (35,36,37). And this content is not always genuine. When there is content, people try to make it fraudulent in various media formats such as internet memes, posts combining text + images and messages combined with visual context. The amount of information spreading is countless, with this much bulk information finding the messages that are genuine is a major challenging task for the owners who are maintaining the social networking sites. For identifying misinformation, if we completely rely on traditional filtering tools then it won't work as this malicious intent not only comes from the images or texts but also through the semantic relationship between these channels (4,5,19,20). This research introduces SEMTEX++, which is an ensemble framework with hybrid clustering, semantics, threat-aware, and explainable generative AI to increase the chance of identifying and analyzing toxic content within social media. Our unified framework produced good performance with an overall accuracy of 91%, F1-score of 0.90, and AUC of 0.95 when compared to text-only, vision-language, and LLM baselines. Results demonstrate that this multimodal fusion approach improves the detection of indirect harmful content, sarcasm,  meme-based hate speech and misinformation. The explanations module achieved the highest faithfulness and human evaluation scores among all evaluated methods.