A Sign-Component-Based Framework for Chinese Sign Language Recognition Using Accelerometer and sEMG Data

Citations Over TimeTop 10% of 2012 papers

Abstract

Identification of constituent components of each sign gesture can be beneficial to the improved performance of sign language recognition (SLR), especially for large-vocabulary SLR systems. Aiming at developing such a system using portable accelerometer (ACC) and surface electromyographic (sEMG) sensors, we propose a framework for automatic Chinese SLR at the component level. In the proposed framework, data segmentation, as an important preprocessing operation, is performed to divide a continuous sign language sentence into subword segments. Based on the features extracted from ACC and sEMG data, three basic components of sign subwords, namely the hand shape, orientation, and movement, are further modeled and the corresponding component classifiers are learned. At the decision level, a sequence of subwords can be recognized by fusing the likelihoods at the component level. The overall classification accuracy of 96.5% for a vocabulary of 120 signs and 86.7% for 200 sentences demonstrate the feasibility of interpreting sign components from ACC and sEMG data and clearly show the superior recognition performance of the proposed method when compared with the previous SLR method at the subword level. The proposed method seems promising for implementing large-vocabulary portable SLR systems.

Related Papers

• → An Efficient Sign Language Recognition (SLR) System Using Camshift Tracker and Hidden Markov Model (HMM)(2021)45 cited
• → Sign Language Gesture Recognition Using HMM(2017)29 cited
• → Live demonstration: A HMM-based real-time sign language recognition system with multiple depth sensors(2015)7 cited
• → A Gesture Recognition Based on Accelerometer and Hidden Markov Model for Human Computer Communication(2013)
• A Survey Paper on Real-Time Sign Language Gesture Recognition(2020)