Elastic functional coding of human actions: From vector-fields to latent variables

Citations Over TimeTop 10% of 2015 papers

Abstract

Human activities observed from visual sensors often give rise to a sequence of smoothly varying features. In many cases, the space of features can be formally defined as a manifold, where the action becomes a trajectory on the manifold. Such trajectories are high dimensional in addition to being non-linear, which can severely limit computations on them. We also argue that by their nature, human actions themselves lie on a much lower dimensional manifold compared to the high dimensional feature space. Learning an accurate low dimensional embedding for actions could have a huge impact in the areas of efficient search and retrieval, visualization, learning, and recognition. Traditional manifold learning addresses this problem for static points in ℝ n , but its extension to trajectories on Riemannian manifolds is non-trivial and has remained unexplored. The challenge arises due to the inherent non-linearity, and temporal variability that can significantly distort the distance metric between trajectories. To address these issues we use the transport square-root velocity function (TSRVF) space, a recently proposed representation that provides a metric which has favorable theoretical properties such as invariance to group action. We propose to learn the low dimensional embedding with a manifold functional variant of principal component analysis (mfPCA). We show that mf-PCA effectively models the manifold trajectories in several applications such as action recognition, clustering and diverse sequence sampling while reducing the dimensionality by a factor of ~ 250×. The mfPCA features can also be reconstructed back to the original manifold to allow for easy visualization of the latent variable space.

Related Papers

• → Guided Locally Linear Embedding(2011)22 cited
• → Principal Components Analysis Competitive Learning(2004)16 cited
• → The Connections between Principal Component Analysis and Dimensionality Reduction Methods of Manifolds(2012)1 cited
• Dimensionality Reduction Algorithm Based on Manifold Learning(2010)
• → Framework of Multiple-point Statistical Simulation Using Manifold Learning for the Dimensionality Reduction of Patterns(2020)