Learning Character-Agnostic Motion for Motion Retargeting in 2D (SIGGRAPH 2019)

News / Lectures/Workshops / Past / Learning Character-Agnostic Motion for Motion Retargeting in 2D (SIGGRAPH 2019)
05 September. 2019 | 11:30 | Julia House, 21612, CY1591, Themistokli Dervi 3, Nicosia 1066

Learning Character-Agnostic Motion for
Motion Retargeting in 2D (SIGGRAPH 2019)

Invited Talk - 5 September, 11:30-12:30


Analyzing human motion is a challenging task with a wide variety of applications in computer vision and in graphics. One such application, of particular importance in computer animation, is the retargeting of motion from one performer to another. While humans move in three dimensions, the vast majority of human motions are captured using video, requiring 2D-to-3D pose and camera recovery, before existing retargeting approaches may be applied. In this talk, we will present a new method for retargeting video-captured motion between different human performers, without the need to explicitly reconstruct 3D poses and/or camera parameters. Our key idea is to train a deep neural network to decompose temporal sequences of 2D poses into three components: motion, skeleton, and camera view-angle. In this way, we learn to extract, directly from a video, a high-level latent motion representation, which is invariant to the skeleton geometry and the camera view. Having extracted such a representation, we are able to re-combine motion with novel skeletons and camera views, and decode a retargeted temporal sequence, which we compare to a ground truth from a synthetic dataset. We demonstrate that our framework can be used to robustly extract human motion from videos, bypassing 3D reconstruction, and outperforming existing retargeting methods, when applied to videos in-the-wild. It also enables additional applications, such as performance cloning, video-driven cartoons, and motion retrieval. 


Kfir is a researcher in the Reality Capture Group at the Advanced InnovationCenter for Future Visual Entertainment (AICFVE) in Beijing. In parallel, Kfir is pursuing his Ph.D in Tel-Aviv University, under the supervision of Prof. Daniel Cohen-Or, where his areas of interests include deep neural networks architectures for computer graphics application. His B.Sc. (summa cum laude) and M.Sc. (cum laude) in Electrical Engineering from the Technion were done under the supervision of Prof. Yonina C. Eldar in the area of signal processing.
More Events
"Disrupting Human-AI Interaction through Language Models: A...
"Disrupting Human-AI Interaction through Language Models: A tour about ChatGPT potentials"
20 July. 2023
15th Cyprus Workshop on Signal Processing and Informatics (CWSPI)
Following the successful one-day workshop we had in the last 14 years
05 July. 2023
Computer Analysis of Images and Patterns (CAIP2023)
CAIP 2023 is the 20th in the CAIP series of biennial international conferences devoted to all...
18 January. 2023
Computer Animation and Social Agents (CASA 2023)
The 36th International Conference onĀ Computer Animation and Social AgentsĀ (CASA 2023) will be...
28 November. 2022