Enhancing Abstractive Text Summarization for Indic Scripts using Transformer-Based Models

Abstract

There has been a rapid growth of digital textual content which has increased the importance of automatic summarization to extract useful information. So far, while it has seen considerable improvement in the case of English and other high-resource languages, summarization in Indian languages still falls far behind compared to these languages due to the diversity, morphological richness and lack of ample annotated dataset. In this work, we have tested how effectively multilingual transformers can perform abstractive summarization for Hindi, Gujarati and Marathi languages. 
The fine-tuned versions of IndicBART and mBART are tested to produce brief summaries with the retention of meanings of the documents. For Hindi and Gujarati, experiments were done with the ILSUM 2.0 benchmark dataset while for Marathi experiments the XL-Sum Marathi corpus was used. The obtained models were evaluated with the evaluation metrics such as Rouge-1, Rouge-2, Rouge-L, BLEU and BERTScore, and compared with the baseline models based on Seq2Seq-LSTM that have widely been used for the summarization tasks on Indic languages. Experimental analysis results show significant improvements across all the evaluated measures. For Hindi language, indicBART gives maximum performance (ROUGE-L, ROUGE-1 and ROUGE-2 values of 0.6176, 0.6449, 0.4763 respectively). 
For the Gujarati language, mBART achieves maximum performance with Rouge-1, Rouge-2 and Rouge-L scores 0.6439, 0.4069 and 0.5899 respectively. 
Experiments on Marathi provide proof that transformer models can be used even in the low-resource situations. In this case, indicBART achieved optimum results. These results prove that language-aware fine-tuning enhance more lexical overlapping, semantic meaning and context conservation capabilities of models.