CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memory
Citations Over TimeTop 10% of 2023 papers
Abstract
We propose CLIP-Fields, an implicit scene model that can be used for a variety of tasks, such as segmentation, instance identification, semantic search over space, and view localization.CLIP-Fields learns a mapping from spatial locations to semantic embedding vectors.Importantly, we show that this mapping can be trained with supervision coming only from webimage and web-text trained models such as CLIP, Detic, and Sentence-BERT; and thus uses no direct human supervision.When compared to baselines like Mask-RCNN, our method outperforms on few-shot instance identification or semantic segmentation on the HM3D dataset with only a fraction of the examples.Finally, we show that using CLIP-Fields as a scene memory, robots can perform semantic navigation in real-world environments.Our code and demonstration videos are available here: https://mahis.life/clip-fields
Related Papers
- Study and Two Types of Typical Usage of DataGrid Web Server Control(2005)
- Achieving Parameter of DBSCAN Based on Datagrid(2010)
- Using DataGrid Control to Realize DataBase of Querying in VB6.0(2000)
- Susquehanna Chorale Spring Concert "Roots and Wings"(2017)
- → DETERMINING QUALITY REQUIREMENTS AT THE UNIVERSITIES TO IMPROVE THE QUALITY OF EDUCATION(2018)