0 references
Attention-Free Keyword Spotting
arXiv (Cornell University)2021
Abstract
Till now, attention-based models have been used with great success in the keyword spotting problem domain. However, in light of recent advances in deep learning, the question arises whether self-attention is truly irreplaceable for recognizing speech keywords. We thus explore the usage of gated MLPs --previously shown to be alternatives to transformers in vision tasks-- for the keyword spotting task. We provide a family of highly efficient MLP-based models for keyword spotting, with less than 0.5 million parameters. We show that our approach achieves competitive performance on Google Speech Commands V2-12 and V2-35 benchmarks with much fewer parameters than self-attention-based methods.
Related Papers
- → A Review of Deep Learning Techniques in Document Image Word Spotting(2021)11 cited
- → Spell My Name: Keyword Boosted Speech Recognition(2022)10 cited
- → Part-of-speech and postion attention mechanism based BLSTM for question answering system(2018)1 cited
- → Position-aware Attention for Enhancing the Machine Comprehension(2018)1 cited
- → Spotting keywords and sensing topic changes in speech(2012)