An Enhanced Hybrid BERT–BiLSTM–GRU Model for Text-Based Emotion Detection

Abstract

Fine-grained emotion detection in user-generated text remains a challenging task because of severe class imbalance and the semantic overlap between related affective categories. This paper investigates whether stacking recurrent layers on top of a pre-trained transformer encoder can improve text-based emotion classification, and under what architectural conditions such stacking is beneficial. We propose an enhanced hybrid model that augments BERT-base with a Bidirectional Long Short-Term Memory (BiLSTM) layer followed by a Bidirectional Gated Recurrent Unit (BiGRU), with classification performed from the [CLS] representation. The model is evaluated on the GoEmotions corpus mapped to the six basic Ekman emotions, and is compared against twelve baselines spanning classical, recurrent, and transformer-based architectures using three random seeds, McNemar's significance tests, and bootstrap-stable mean ± standard deviation reporting. The proposed model attains the highest mean macro-F1 (0.7233 ± 0.0042), exceeding BERT-base by 0.93 percentage points (p < 0.05, Bonferroni-corrected) and matching RoBERTa-base, which uses substantially more pre-training data, despite a 9.5% smaller parameter budget. Architectural ablation shows that all three design choices—the BiLSTM layer, the BiGRU layer, and CLS-based pooling—are individually necessary: removing any one of them reduces macro-F1 to below the BERT-base baseline. Per-class analysis reveals heterogeneous trade-offs, with significant gains on the majority class and on surprise but a regression on sadness relative to several transformer baselines. The model also incurs a 2.7× single-sample inference latency penalty over BERT-base, which is largely amortised at batch inference. The findings clarify when and why recurrent stacking on top of transformers is useful, and provide an honest, fully reproducible empirical baseline for future work on transformer–RNN hybrids in emotion classification.