Infusing behavior science into large language models for activity coaching

Citations Over TimeTop 10% of 2024 papers

Abstract

Large language models (LLMs) have shown promise for task-oriented dialogue across a range of domains. The use of LLMs in health and fitness coaching is under-explored. Behavior science frameworks such as COM-B, which conceptualizes behavior change in terms of capability (C), Opportunity (O) and Motivation (M), can be used to architect coaching interventions in a way that promotes sustained change. Here we aim to incorporate behavior science principles into an LLM using two knowledge infusion techniques: coach message priming (where exemplar coach responses are provided as context to the LLM), and dialogue re-ranking (where the COM-B category of the LLM output is matched to the inferred user need). Simulated conversations were conducted between the primed or unprimed LLM and a member of the research team, and then evaluated by 8 human raters. Ratings for the primed conversations were significantly higher in terms of empathy and actionability. The same raters also compared a single response generated by the unprimed, primed and re-ranked models, finding a significant uplift in actionability and empathy from the re-ranking technique. This is a proof of concept of how behavior science frameworks can be infused into automated conversational agents for a more principled coaching experience.

Related Papers

• → Empathy has not been measured in clients’ terms or effectively taught: a review of the literature(1999)78 cited
• → Toward a Holistic Conceptualization of Empathy for Nursing Practice(2007)37 cited
• → A study of empathy in student nurses(1988)39 cited
• بررسی یکدلی (Empathy) در میان دانشجویان گروه پزشکی: مطالعه مروری(2016)
• → The Empathy Tank as a revised model for fostering empathy in medical education(2018)