Behavior Retrieval: Few-Shot Imitation Learning by Querying Unlabeled Datasets

Citations Over TimeTop 10% of 2023 papers

Abstract

Enabling robots to learn novel visuomotor skills in a data-efficient manner remains an unsolved problem with myriad challenges.A popular paradigm for tackling this problem is through leveraging large unlabeled datasets that have many behaviors in them and then adapting a policy to a specific task using a small amount of task-specific human supervision (i.e.interventions or demonstrations).However, how best to leverage the narrow task-specific supervision and balance it with offline data remains an open question.Our key insight in this work is that task-specific data not only provides new data for an agent to train on but can also inform the type of prior data the agent should use for learning.Concretely, we propose a simple approach that uses a small amount of downstream expert data to selectively query relevant behaviors from an offline, unlabeled dataset (including many sub-optimal behaviors).The agent is then jointly trained on the expert and queried data.We observe that our method learns to query only the relevant transitions to the task, filtering out sub-optimal or task-irrelevant data.By doing so, it is able to learn more effectively from the mix of task-specific and offline data compared to naïvely mixing the data or only using the task-specific data.Furthermore, we find that our simple querying approach outperforms more complex goal-conditioned methods by 20% across simulated and real robotic manipulation tasks from images.See https://sites.google.com/view/behaviorretrievalfor videos and code.

Related Papers

• → Improved Few-Shot Learning for Images Classification(2022)2 cited
• One Shot Device의 저장 신뢰도 분석에 관한 연구(2016)
• Methods for Determination of Gun Shot Residues(2011)
• Does Use of Steel Shot Result in Greater Waterfowl Crippling? Using Harvest Data to Determine Effectiveness of Non-toxic Shot(2017)
• I've heard that there's a high-dose flu shot available. Is it better for older adults than the regular flu shot? Should I ask for the high-dose shot?(2011)