Toward User-Driven Sound Recognizer Personalization with People Who Are d/Deaf or Hard of Hearing

Citations Over TimeTop 10% of 2021 papers

Abstract

Automated sound recognition tools can be a useful complement to d/Deaf and hard of hearing (DHH) people's typical communication and environmental awareness strategies. Pre-trained sound recognition models, however, may not meet the diverse needs of individual DHH users. While approaches from human-centered machine learning can enable non-expert users to build their own automated systems, end-user ML solutions that augment human sensory abilities present a unique challenge for users who have sensory disabilities: how can a DHH user, who has difficulty hearing a sound themselves, effectively record samples to train an ML system to recognize that sound? To better understand how DHH users can drive personalization of their own assistive sound recognition tools, we conducted a three-part study with 14 DHH participants: (1) an initial interview and demo of a personalizable sound recognizer, (2) a week-long field study of in situ recording, and (3) a follow-up interview and ideation session. Our results highlight a positive subjective experience when recording and interpreting training data in situ, but we uncover several key pitfalls unique to DHH users---such as inhibited judgement of representative samples due to limited audiological experience. We share implications of these results for the design of recording interfaces and human-the-the-loop systems that can support DHH users to build sound recognizers for their personal needs.

Related Papers

• Robotics research : The 13th International Symposium ISRR(2010)
• → INVITED SESSION, SESSION 32: LIVE SURGERY SESSION, Tuesday 5 July 2011 10:00 - 13:00(2011)1 cited
• → Session details: Session 8: ISPD 2020 Contest Results and Poster Presentations(2020)
• → IUPsyS Invited Symposium, Poster Session, Oral Session, Keynote, State‐of‐the‐Art Session(2004)
• Proceedings of PART '98 : the 5th Australasian conference on Parallel and Real-Time Systems, University of Adelaide, Adelaide, Australia, 28-29 September 1998(1998)