Bridging the Communication Gap: An AI-Driven word-level Bangla Sign Language Interpreter

Abstract

The present paper is a research project report creating a real-time Bangla Sign Language (BdSL) recognition system in a two-phase approach. The initial phase concentrated on developing a state-of-the-art baseline with our proposed hybrid Conv1D-BiLSTM architecture including the Attention mechanism with an accuracy of 97.3% on the published 60-word BDSLw60 v2 dataset, which is higher than the previously published benchmarks. The second stage of research was devoted to the important issue of vocabulary development, and a new set of signs of 39 signs, which had been collected by hand, was presented. In order to address the dramatic accuracy drop experienced with first fine-tuning, we designed and adopted a professional quality multi-stage fine-tuning policy that includes class weights and gradual unfreezing. This sophisticated procedure was very practical and the final and strong model reached 93 percent total accuracy in the whole vocabulary (99-word). This last model is implemented on a real-time Flask web application, which translates signs directly to text, showing an end-to-end cycle between a validation of the baseline to a scalable and real-world application.