Catalytic Role Of Noise And Necessity Of Inductive Biases In The Emergence Of Compositional Communication

Citations Over Time

Abstract

Communication is compositional if complex signals can be represented as a combination of simpler subparts. In this paper, we theoretically show that inductive biases on both the training framework and the data are needed to develop a compositional communication. Moreover, we prove that compositionality spontaneously arises in the signaling games, where agents communicate over a noisy channel. We experimentally confirm that a range of noise levels, which depends on the model and the data, indeed promotes compositionality. Finally, we provide a comprehensive study of this dependence and report results in terms of recently studied compositionality metrics: topographical similarity, conflict count, and context independence.

Related Papers

• → Compositionality for quantitative specifications(2017)3 cited
• On-line learning of predictive compositional hierarchies(2002)
• The compositionality of neural networks: integrating symbolism and connectionism.(2019)
• → Inductive Biases and Variable Creation in Self-Attention Mechanisms(2021)15 cited
• A Framework for Measuring Compositional Inductive Bias.(2021)