Dialogue systems for language learning: A meta-analysis

Citations Over TimeTop 10% of 2022 papers

Abstract

The present study offers a meta-analysis of effectiveness studies on dialogue-based CALL, systems affording a learner practice in a foreign language (L2) by interacting with a conversational agent (“bot”). Through a systematic inclusion and exclusion process, we identified 17 relevant meta-analyzable studies. We made use of Morris and DeShon’s (2002) formulas to compute comparable effect sizes across designs, including k = 100 individual effect sizes, which were analyzed through a multilevel random-effects model. Results confirm that dialogue-based CALL practice had a significant medium effect size on L2 proficiency development (d = 0.58). We performed extensive moderator analyses to explore the relative effectiveness on several learning outcomes of different types and features of dialogue-based CALL (type of interaction, modality, constraints, feedback, agent embodiment, gamification). Our study confirms the effectiveness of form-focused and goal-oriented systems, system-guided interactions, corrective feedback provision, and gamification features. Effects for lower proficiency learners, and on vocabulary, morphosyntax, holistic proficiency, and accuracy are established. Finally, we discuss expected evolutions in dialogue-based CALL and the language learning opportunities it offers.

Related Papers

• → Few-Shot Semantic Parsing with Language Models Trained on Code(2022)37 cited
• → First Language Acquisition: Learning or Spontant(2021)2 cited
• → Research on the Application of Learning Frequency in Cultivating Language Sense in Second Language Acquisition(2023)1 cited
• Semantic Parsing of Brief and Multi-Intent Natural Language Utterances(2021)
• → Few-Shot Semantic Parsing with Language Models Trained On Code(2021)