A Conversational Agent Framework with Multi-modal Personality Expression

Citations Over TimeTop 10% of 2021 papers

Abstract

Consistently exhibited personalities are crucial elements of realistic, engaging, and behavior-rich conversational virtual agents. Both nonverbal and verbal cues help convey these agents’ unseen psychological states, contributing to our effective communication with them. We introduce a comprehensive framework to design conversational agents that express personality through non-verbal behaviors like body movement and facial expressions, as well as verbal behaviors like dialogue selection and voice transformation. We use the OCEAN personality model, which defines personality as a combination of five orthogonal factors of openness, conscientiousness, extraversion, agreeableness, and neuroticism. The framework combines existing personality expression methods with novel ones such as new algorithms to convey Laban Shape and Effort qualities. We perform Amazon Mechanical Turk studies to analyze how different communication modalities influence our perception of virtual agent personalities and compare their individual and combined effects on each personality dimension. The results indicate that our personality-based modifications are perceived as natural, and each additional modality improves perception accuracy, with the best performance achieved when all the modalities are present. We also report some correlations for the perception of conscientiousness with neuroticism and openness with extraversion.

Related Papers

• The investigation of personality characteristics in athlete andnon-athlete students(2013)
• → Neuroticism, Agreeableness, and Conscientiousness and the Relationship with Individual Taxpayer Compliance Behavior(2019)12 cited
• → Football Scores on the Big Five Personality Factors across 50 States in the U.S.(2012)11 cited
• → Agreeableness and Conscientiousness as predictors of university students’ self/peer-assessment rating error(2016)6 cited
• Football Scores on the Big Five Personality Factors across 50 States in the(2012)