Attention-Based Sequence-to-Sequence Neural Machine Translation from English to Bangla

Abstract

Neural Machine Translation (NMT) has emerged as a dominant form of automatic languages, especially in the case of lower-resourced languages like Bengali. We developed an English-to-Bengali translation system utilizing the BanglaT5 Transformer model, which was pre-trained on Bengali data. The parallel English-Bengali dataset was cleaned of unwanted characters, to the best of our ability, and normalized for a large enough data pool to train the model, and to ensure uniformity. The sentences in the dataset were tokenized to prepare the input sequences and attention masks, with associated target labels, with the aim of supervised learning. A custom-made dataset with DataLoader from Pytorch allowed the batch size to be tailored and to also facilitate efficient training on the GPU. The BanglaT5 model was fine-tuned using cross-entropy loss with the Adam optimizer to minimize the loss function using backpropagation. Evaluation was performed using BLEU and chrF scores to assess translation accuracy of Bengali sentences generated by the modelagainst provided reference translations. A good illustration of reining in of this system is evident. With great success on a low-resourced NMT problem, a pre-trained Transformer is used. translation. As outlined in this paper, appropriate texts normalizing, prep pre tokenizing, and fine-tuning are important to eventually come up with superior quality Machine Translation. System (MTS). This is a strategy that has been scaled to. host other languages with low resources and is a resourceful guide. South-Asian language-pairs NMT tasks implementation.