Emotion Recognition Based on Joint Visual and Audio Cues

Citations Over TimeTop 10% of 2006 papers

Abstract

Recent technological advances have enabled human users to interact with computers in ways previously unimaginable. Beyond the confines of the keyboard and mouse, new modalities for human-computer interaction such as voice, gesture, and force-feedback are emerging. However, one necessary ingredient for natural interaction is still missing - emotions. This paper describes the problem of bimodal emotion recognition and advocates the use of probabilistic graphical models when fusing the different modalities. We test our audio-visual emotion recognition approach on 38 subjects with 11 HCI-related affect states. The experimental results show that the average person-dependent emotion recognition accuracy is greatly improved when both visual and audio information are used in classification.

Related Papers

• → Touch Me, Tilt Me – Comparing Interaction Modalities for Navigation in 2D and 3D Worlds on Mobiles(2013)4 cited
• → Exploring Gesture-Based Tangible Interactions with a Lighting AI Agent(2019)4 cited
• → Incorporating Tilt-Based Interaction in Multimodal User Interfaces for Mobile Devices(2007)2 cited
• → Is Natural User Interaction Really Natural? An evaluation of gesture-based navigating techniques in Virtual Environments(2018)2 cited
• → Real-Time Bimanual Interaction Across Virtual Workspaces(2022)1 cited