Perceptual-IQ: Visual Commonsense Reasoning about Perceptual Imagination
2022 IEEE International Conference on Big Data (Big Data)2022pp. 6581–6583
Citations Over Time
Abstract
In this paper, we present a new dataset Perceptual Imagination: Question-Answering (Perceptual-IQ) to evaluate the visual systems’ commonsense reasoning ability when confronted with perceptual changes. In our dataset, the machines are given a question that includes a perceptual change over an image and they have to predict human response to the change. Perceptual-IQ consists of 3.7K manually annotated QA pairs from 1.6K curated images and covers various types of perceptual changes. Through the evaluation of vision-language models with Perceptual-IQ, we identify the performance gap (~25%) with human performance.
Related Papers
- → Beating Common Sense into Interactive Applications(2004)148 cited
- → UFO: Unified Fact Obtaining for Commonsense Question Answering(2023)1 cited
- → CapableOf Reasoning: A Step Towards Commonsense Oracle(2020)1 cited
- → Benchmarks for Automated Commonsense Reasoning: A Survey(2023)6 cited
- An Interface for Crowd-sourcing Spatial Models of Commonsense(2011)