PlaneMVS: 3D Plane Reconstruction from Multi-View Stereo

Citations Over TimeTop 10% of 2022 papers

Abstract

We present a novel framework named PlaneMVS for 3D plane reconstruction from multiple input views with known camera poses. Most previous learning-based plane reconstruction methods reconstruct 3D planes from single images, which highly rely on single-view regression and suffer from depth scale ambiguity. In contrast, we reconstruct 3D planes with a multi-view-stereo (MVS) pipeline that takes advantage of multi-view geometry. We decouple plane reconstruction into a semantic plane detection branch and a plane MVS branch. The semantic plane detection branch is based on a single-view plane detection framework but with differences. The plane MVS branch adopts a set of slanted plane hypotheses to replace conventional depth hypotheses to perform plane sweeping strategy and finally learns pixel-level plane parameters and its planar depth map. We present how the two branches are learned in a balanced way, and propose a soft-pooling loss to associate the outputs of the two branches and make them benefit from each other. Extensive experiments on various indoor datasets show that PlaneMVS significantly outperforms state-of-the-art (SOTA) single-view plane reconstruction methods on both plane detection and 3D geometry metrics. Our method even outperforms a set of SOTA learning-based MVS methods thanks to the learned plane priors. To the best of our knowledge, this is the first work on 3D plane reconstruction within an end-to-end MVS framework.

Related Papers

• Pooling in high-throughput drug screening.(2009)
• → A fully trainable network with RNN-based pooling(2019)22 cited
• → Why and when should you pool? Analyzing Pooling in Recurrent Architectures(2020)1 cited
• Alpha-Pooling for Convolutional Neural Networks.(2018)
• → A Fully Trainable Network with RNN-based Pooling(2017)1 cited