Sentiment Analysis in Nepali Tweets: Leveraging TransformerBased Pre-trained Models
- Despite the remarkable achievements of large transformer-based pre-trained models like BERT, GPT in several Natural Language Processing (NLP) tasks including Sentiment Analysis (SA), challenges are still present for subdued source languages like Nepali. Nepali language is written in Devanagari script, has complex grammatical structure and diverse linguistic features. Due to the absence of balanced datasets, and computational resources for Nepali, achieving optimal result with the latest architecture remains challenging. For this reason, publicly available NLP modelsfor Nepali are very less, making research in this area difficult. This paper attempts to addressthis gap through the use of pre-trained transformer models specially tailored for Nepali from Hugging Face including BERT, DistilBERT, ALBERT, and DeBERTa for sentiment analysis in Nepali tweets on relatively balanced datasets. The models are trained on large Nepali datasets and optimized for NLP tasks involving Devanagari scripts. To evaluate the model’s performance, various tokenization strategies are investigated in order to capitalize on transformer-based embedding with the SoftMax function and confusion matrix. The outcomes of models are compared using the same datasets. The study’s results shows that DistilBERT achieved the highest accuracy rate of 88% in Nepali sentiment analysis tasks, followed by BERT and DeBERTa at 83% and 80%, respectively. However, ALBERT showed a low accuracy of 70%. The result of this approach shares valuable viewpoints for the field of sentiment analysis in diverse linguistic contexts.