Deepsign: Sign Language Detection and Recognition Using Deep Learning

Citations Over TimeTop 1% of 2022 papers

Abstract

The predominant means of communication is speech; however, there are persons whose speaking or hearing abilities are impaired. Communication presents a significant barrier for persons with such disabilities. The use of deep learning methods can help to reduce communication barriers. This paper proposes a deep learning-based model that detects and recognizes the words from a person’s gestures. Deep learning models, namely, LSTM and GRU (feedback-based learning models), are used to recognize signs from isolated Indian Sign Language (ISL) video frames. The four different sequential combinations of LSTM and GRU (as there are two layers of LSTM and two layers of GRU) were used with our own dataset, IISL2020. The proposed model, consisting of a single layer of LSTM followed by GRU, achieves around 97% accuracy over 11 different signs. This method may help persons who are unaware of sign language to communicate with persons whose speech or hearing is impaired.

Related Papers

• → Real‐time hand gestures system based on leap motion(2018)11 cited
• → A Review of Sign Language Hand Gesture Recognition Algorithms(2020)4 cited
• → Methods to describe and recognize sign language based on gesture components represented by symbols and numerical values(1998)10 cited
• → Systematic Literature Survey on Sign Language Recognition Systems(2022)
• → Gesture Vocalizer for Assisting Deaf and Dumb(2023)