Vision-based Real Time Bangla Sign Language Recognition System Using MediaPipe Holistic and LSTM

Abstract

The Bangla Sign Language Detection System converts the Bangla sign language into text so that deaf-mute people can communicate with ordinary people. Deaf-mute people are quite detached from society because normal individuals and deaf- mute individuals have a communication gap. On the strength of technological welfare, it is now possible to capture any deaf- mute person’s gesture, and with the help of machine learning, it can be converted into text. In this research, we adopt a development model for recognizing gestures that accommodates MediaPipe for extracting hands and posing landmarks and long short-term memory (LSTM) to train and recognize the gesture. This will convert Bangla sign language gestures into readable text. The requirements analysis served as the foundation for our proposed model, which will be carried out in four stages: collecting data and processing it, training and testing the chosen neural network, and finally, real-time testing. A gesture model is taught to recognize gestures with the help of a self-created dataset of Bangla sign language. The trained model successfully identifies the gesture, and the text equivalent of the gesture is displayed on screen. The purpose of this model is to help create a medium to communicate between normal people and the hearing impaired.