Knowledge-Enriched Visual Storytelling

Citations Over TimeTop 10% of 2020 papers

Abstract

Stories are diverse and highly personalized, resulting in a large possible output space for story generation. Existing end-to-end approaches produce monotonous stories because they are limited to the vocabulary and knowledge in a single training dataset. This paper introduces KG-Story, a three-stage framework that allows the story generation model to take advantage of external Knowledge Graphs to produce interesting stories. KG-Story distills a set of representative words from the input prompts, enriches the word set by using external knowledge graphs, and finally generates stories based on the enriched word set. This distill-enrich-generate framework allows the use of external resources not only for the enrichment phase, but also for the distillation and generation phases. In this paper, we show the superiority of KG-Story for visual storytelling, where the input prompt is a sequence of five photos and the output is a short story. Per the human ranking evaluation, stories generated by KG-Story are on average ranked better than that of the state-of-the-art systems. Our code and output stories are available at https://github.com/zychen423/KE-VIST.

Related Papers

• → Storytelling is not just for marketing: Cultivating a storytelling culture throughout the organization(2023)41 cited
• → Storytellers and Storytelling in Northeast Thailand(2008)4 cited
• → Review of Storytelling in Organizations: Why Storytelling Is Transforming 21st Century Organizations and Management by John Seeley Brown(2006)2 cited
• → Review of Storytelling in Daily Life: Performing Narrative by Langellier and Peterson(2006)1 cited
• Film Story: Telling & Showing(2018)