ISL Sign Language Recognition Using LSTM-Driven Deep Learning Model

Abstract

In the development of the "ISL Sign Language Recognition Using LSTM-Driven Deep Learning Model," our methodology integrates advanced computer vision and deep learning techniques to achieve high accuracy in gesture recognition. The system leverages MediaPipe, an open-source library developed by Google, which facilitates real-time hand tracking and gesture detection. MediaPipe provides a reliable framework for extracting precise spatial information from video frames, identifying key landmarks on the hands, and distinguishing complex movements associated with Indian Sign Language (ISL). This pre-processing step ensures that the raw video data is converted into structured data, capturing the essential features necessary for gesture recognition. The processed data is then fed into a Long Short-Term Memory (LSTM) network, a type of recurrent neural network (RNN) well-suited for sequential data. LSTM networks are designed to address the vanishing gradient problem common in traditional RNNs, making them ideal for modeling temporal dependencies. By using the LSTM's unique memory cell structure, the system effectively retains information over longer sequences, which is critical for understanding the continuity and nuances of hand movements in sign language. The training phase employed a dataset of commonly used ISL signs. Each sign was captured in multiple video samples to ensure variability and robustness in the model. The LSTM model was trained to classify the sequential hand gestures, adjusting weights through backpropagation and optimizing the network using stochastic gradient descent. During training, data augmentation techniques were applied to enhance the model's ability to generalize, preventing overfitting and increasing accuracy. The model's performance was evaluated through a series of tests, measuring precision, recall, and overall accuracy. The results demonstrated that the LSTM-driven model outperforms traditional methods by effectively capturing and interpreting intricate hand movements, achieving superior recognition rates. This research contributes to assistive technology by improving the accessibility of communication for individuals with hearing impairments and promoting inclusivity in society.