Online non-rigid motion and scene layer segmentation

Abstract

In the past, different kinds of methods were devised to detect objects from videos. Based on the assumption of stationary camera, the now ubiquitous background subtraction learns the appearance of the background and then subtracts it to segment the scene. In practice such assumption is highly restrictive, and to handle moving cameras other methods were devised. For instance, motion segmentation targets the segmentation of different rigid motions in the video, while scene layer segmentation attempts to find a segmentation of the scene into layers that are consistent in space and time. Yet, such methods still suffers from other limitations such as the requirement of point trajectories to span the entire frame sequence. On a different aspect, recent years have witnessed a large increase in the proportion of videos coming from streaming sources such as TV Broadcast, Internet video streaming, and streaming from mobile devices. Unfortunately, most methods that process videos are mainly offline and with a high computational complexity. Thus rendering them ineffective for processing videos from streaming sources. This highlights the need for novel techniques that are online and efficient at the same time. In this dissertation, we first generalize motion segmentation by showing that under a general perspective camera trajectories belonging to one moving object form a low-dimensional manifold. Based on this, we devise two methods for online nonrigid motion segmentation. The first method tries to explicitly reconstruct the low-dimensional manifolds and then cluster them. The second method attempts to directly separate the manifolds. We then show how motion segmentation and scene layer segmentation can be combined in a single online framework that combines the strength of both approaches. Finally, we propose two methods that assign figure- ground labels to layers by combining several cues. Results show that our framework is effective in detecting moving objects from videos captured by a moving camera.

Related Papers

• → Video Pop-up: Monocular 3D Reconstruction of Dynamic Scenes(2014)103 cited
• Scene understanding with complete scenes and structured representations(2014)
• → The Video Scene(2009)
• VIDEO SCENE SEGMENTATION TO SEPARATE SCRIPT(2015)