Structured World Models from Human Videos
Citations Over TimeTop 10% of 2023 papers
Abstract
In this paper, we tackle the problem of learning complex, general behaviors directly in the real world.We propose an approach for robots to efficiently learn manipulation skills using only a handful of real-world interaction trajectories from many different settings.Inspired by the success of learning from large-scale datasets in the fields of computer vision and natural language, our belief is that in order to efficiently learn, a robot must be able to leverage internet-scale, human video data.Humans interact with the world in many interesting ways, which can allow a robot to not only build an understanding of useful actions and affordances but also how these actions affect the world for manipulation.Our approach builds a structured, human-centric action space grounded in visual affordances learned from human videos.Further, we train a world model on human videos and finetune on a small amount of robot interaction data without any task supervision.We show that this approach of affordance-space world models enables different robots to learn various manipulation skills in complex settings, in under 30 minutes of interaction.Videos can be found at https://human-world-model.github.io
Related Papers
- Study and Two Types of Typical Usage of DataGrid Web Server Control(2005)
- Achieving Parameter of DBSCAN Based on Datagrid(2010)
- Using DataGrid Control to Realize DataBase of Querying in VB6.0(2000)
- Susquehanna Chorale Spring Concert "Roots and Wings"(2017)
- → DETERMINING QUALITY REQUIREMENTS AT THE UNIVERSITIES TO IMPROVE THE QUALITY OF EDUCATION(2018)