American sign language recognition with the kinect
Citations Over TimeTop 1% of 2011 papers
Abstract
We investigate the potential of the Kinect depth-mapping camera for sign language recognition and verification for educational games for deaf children. We compare a prototype Kinect-based system to our current CopyCat system which uses colored gloves and embedded accelerometers to track children's hand movements. If successful, a Kinect-based approach could improve interactivity, user comfort, system robustness, system sustainability, cost, and ease of deployment. We collected a total of 1000 American Sign Language (ASL) phrases across both systems. On adult data, the Kinect system resulted in 51.5% and 76.12% sentence verification rates when the users were seated and standing respectively. These rates are comparable to the 74.82% verification rate when using the current(seated) CopyCat system. While the Kinect computer vision system requires more tuning for seated use, the results suggest that the Kinect may be a viable option for sign verification.
Related Papers
- → Model for Recognition of Sign Language and Hand Gesture by Hard Hearing People(2023)3 cited
- → A Review of Sign Language Hand Gesture Recognition Algorithms(2020)4 cited
- → Code Switching in Deaf Adults(1987)10 cited
- Towards a Sign Language Hand Gesture Recognition Design Framework(2020)
- → The automation of spinal non-manual signals in American Sign Language(2011)