Object and Scene Synthesis via deep neural networks

Synthesizing 3D objects and 3D scenes has received a great deal of attention in recent years due to its applications in simulation, AI, robotics and 3D modelling. Recent work has taken advantage of deep generative networks such as variational autoencoders (VAEs) and generative adversarial networks (GANs) which have found great success in generating 2D images or mapping input to output images (e.g. for altering image style). Extending these ideas to work on 3D data is far from trivial, with very recent efforts focused on GANs operating on volumetric representations or latent-GANs operating on point cloud latent spaces learned by autoencoders. 

Basic knowledge of computer vision and deep learning. Programming skills: python, TensorFlow (optional), PyTorch (optional) 

Intermediate

This project proposes applying architectures such as latent-GANs to the problem of 3D object and scene synthesis, tackling challenges such as increasing the realism of the synthesized output as well as the scalability of the proposed methods. 

Expected deliverables: Final report, trained networks, code base