Abstract:
This project presents a comparative study on Bangla news headline generation using
two transformer-based models: the multilingual mT5 and the monolingual BT5-base.
Aimed at addressing the scarcity of effective headline generation tools for low-
resource languages like Bangla, the study evaluates both models on a curated
dataset using standard performance metrics. While both models demonstrated
stable training behavior, BT5-base exhibited faster convergence and lower validation
loss, indicating more efficient learning. Evaluation results reveal a stark contrast in
output quality: BT5-base achieved a ROUGE-1 F1 score of over 56% and a ROUGE-
2 score of 45.92%, significantly outperforming mT5, whose scores remained below 3%
across all ROUGE metrics. Furthermore, BT5-base attained a 21.33% exact match
rate and showed markedly lower Character Error Rate (CER) and Word Error Rate
(WER), highlighting its superior ability to produce semantically and lexically aligned
headlines. These results affirm the effectiveness of domain-specific pretraining, as
the Bangla-focused BT5-base consistently delivered more fluent, accurate, and
culturally appropriate headlines than the multilingual mT5 model. The findings
underscore the value of monolingual transformer models for text generation in
underrepresented languages and contribute a practical foundation for future
advancements in Bangla NLP applications.
