AMTF-Net: An Adaptive Multi-Task Transformer Fusion Network for Joint Sentiment Analysis and Sarcasm Detection

Abstract

With the widespread use of social media platforms, there is a massive amount of user-generated text which necessitates performing sentiment analysis, a natural language processing task. Nevertheless, correctly identifying the sentiment proves to be a difficult task because of the occurrence of sarcasm where the literal sentiment is opposite to the intended meaning. Prior research has either addressed sentiment analysis and sarcasm detection as separate tasks or developed static ensemble methods that do not exploit the semantic relationship between the two tasks. To overcome these drawbacks, this work presents AMTF-Net (Adaptive Multi-Task Transformer Fusion Network), a unified multi-task learning framework capable of jointly performing sentiment analysis and sarcasm detection.  The framework extracts complementary contextual representations using BERT, GPT-2, and XLNet. The extracted contextual representations are fused through a MATF module that adaptively learns the contribution of each transformer for every input. The Shared Semantic Encoder (SSE) and Cross-Task Attention Interaction (CTAI) module refine the fused features to allow knowledge sharing between the two tasks. A Dynamic Joint Loss (DJL) function is used, which adaptively balances the optimization objectives of both tasks. The three public benchmark datasets employed were Twitter, Reddit, and HuffPost news headlines. The experimental results show that AMTF-Net has superior performance against individual transformers, conventional ensemble methods, and recent state-of-the-art methods. The proposed framework was evaluated on Twitter, Reddit, and HuffPost benchmark datasets for joint sentiment analysis and sarcasm detection. The proposed framework achieved an accuracy of 97.74 %, 98.12 % and 98.05 % respectively. Also, the Macro F1-scores were achieved at 98.12 %, 98.04 % and 97.82 % respectively for each dataset for sarcasm detection. These findings show that adaptive transformer fusion and cross-task semantic interaction enhance the robustness and accuracy of joint sentiment analysis and sarcasm detection.