Self-Supervised Pretraining of 3D Features on any Point-Cloud

Citations Over TimeTop 1% of 2021 papers

Abstract

Pretraining on large labeled datasets is a prerequisite to achieve good performance in many computer vision tasks like image recognition, video understanding etc. However, pretraining is not widely used for 3D recognition tasks where state-of-the-art methods train models from scratch. A primary reason is the lack of large annotated datasets because 3D data labelling is time-consuming. Recent work shows that self-supervised learning is useful to pretrain models in 3D but requires multi-view data and point correspondences. We present a simple self-supervised pretraining method that can work with single-view depth scans acquired by varied sensors, without 3D registration and point correspondences. We pretrain standard point cloud and voxel based model architectures, and show that joint pretraining further improves performance. We evaluate our models on 9 benchmarks for object detection, semantic segmentation, and object classification, where they achieve state-of-the-art results. Most notably, we set a new state-of-the-art for object detection on ScanNet (69.0% mAP) and SUNRGBD (63.5% mAP). Our pretrained models are label efficient and improve performance for classes with few examples.

Related Papers

• → Divided Voxels: an efficient algorithm for interactive cutting of deformable objects(2020)11 cited
• → Weighted voxel(2018)9 cited
• → <title>Volume visualization using relative distance among voxels</title>(2002)8 cited
• → Surface scan voxel coloring: A surface scan algorithm for scene reconstruction by voxel coloring(2010)
• → Line-Based Structure for Voxel Model(2021)