Robust motion in-betweening

Citations Over TimeTop 1% of 2020 papers

Abstract

In this work we present a novel, robust transition generation technique that can serve as a new tool for 3D animators, based on adversarial recurrent neural networks. The system synthesises high-quality motions that use temporally-sparse keyframes as animation constraints. This is reminiscent of the job of in-betweening in traditional animation pipelines, in which an animator draws motion frames between provided keyframes. We first show that a state-of-the-art motion prediction model cannot be easily converted into a robust transition generator when only adding conditioning information about future keyframes. To solve this problem, we then propose two novel additive embedding modifiers that are applied at each timestep to latent representations encoded inside the network's architecture. One modifier is a time-to-arrival embedding that allows variations of the transition length with a single model. The other is a scheduled target noise vector that allows the system to be robust to target distortions and to sample different transitions given fixed keyframes. To qualitatively evaluate our method, we present a custom MotionBuilder plugin that uses our trained model to perform in-betweening in production scenarios. To quantitatively evaluate performance on transitions and generalizations to longer time horizons, we present well-defined in-betweening benchmarks on a subset of the widely used Human3.6M dataset and on LaFAN1, a novel high quality motion capture dataset that is more appropriate for transition generation. We are releasing this new dataset along with this work, with accompanying code for reproducing our baseline results.

Related Papers

• → Embedding as a modeling problem(1998)167 cited
• → Keyframe animation and moition capture for creating animation: a survey and perception from industry people(2005)6 cited
• → Japanese Experimental Animation in 1960s(2012)
• → From 1950s to 1980s: The Formation, Development and Practice of the “National Style” Animation Concept of Shanghai Animation Film Studio(2022)
• → Η αξιοποίηση του animation και της εξ αποστάσεως εκπαίδευσης στην διαμόρφωση οδηγών προγραμμάτων χρηματοδότησης στον τομέα του πολιτισμού-οπτικοακουστικά μέσα(2022)