CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
Citations Over TimeTop 1% of 2017 papers
Abstract
When building artificial intelligence systems that can reason and answer questions about visual data, we need diagnostic tests to analyze our progress and discover short-comings. Existing benchmarks for visual question answering can help, but have strong biases that models can exploit to correctly answer questions without reasoning. They also conflate multiple sources of error, making it hard to pinpoint model weaknesses. We present a diagnostic dataset that tests a range of visual reasoning abilities. It contains minimal biases and has detailed annotations describing the kind of reasoning each question requires. We use this dataset to analyze a variety of modern visual reasoning systems, providing novel insights into their abilities and limitations.
Related Papers
- → Qualitative spatial representation and reasoning techniques(1997)183 cited
- Modeling and Comparing Farm Maps using Graphs and Case-based Reasoning(2003)
- → Combining Qualitative Spatial Reasoning and Ontological Reasoning for Supporting Robot Tasks(2012)2 cited
- → Visual reasoning and representation(2007)
- Towards a unified reasoning theory: An evaluation of the Human Reasoning Module in Spatial Reasoning(2015)