Sergey Tulyakov

Welcome! I am a Lead Research Scientist heading the Creative Vision team at Snap Research. My work focuses on creating methods for manipulating the world via computer vision and machine learning. This includes style transfer, photorealistic object manipulation and animation, video synthesis, prediction and retargeting. My work has been published as 20+ top conference papers, journals and patents resulting in multiple tech transfers, including Snapchat Pet Tracking and Real-time Neural Lenses (gender swap, baby face).

Before joining Snap Inc., I built the real-time face tracking and reconstruction engine behind facify.me. 2016, I was a research intern at Microsoft Research, Cambridge, UK, and worked with Sebastian Nowozin and Andrew Fitzgibbon on improving generative models. In 2017, I interned at NVIDIA with Ming-Yu Liu, Xiaodong Yang and Jan Kautz and worked on video generation.

I am a PhD graduate from the University of Trento, Italy, where I worked with Nicu Sebe at the Multimedia and Human Understanding Group MHUG. Before joining the MHUG group, I worked as a software engineer for six years.

I am looking for prospective interns and collaborators. If you are interested send me an email.

Media coverage:

Contact: stulyakov@snap.com


Download CV

The ultimate goal of the Creative Vision group is to develop cutting edge computer vision, machine learning, and graphics technology to unlock creativity of users and creators in a variety of products at Snap. Our work enables machines to see, understand and manipulate the world. We focus on the following broad directions:

  • Understanding Humans: Human digitization (segmentation, 3D pose and geometry of face, body, hand, clothes)
  • Understanding the World: Digitization of the world and objects in it (recognition, segmentation, 3D reconstruction, scene understanding)
  • Tools for Creativity: Arbitrary AR manipulation of the world, including style transfer for shape and texture of faces, objects and scenes, adding and removing objects from the scene
  • Efficient Algorithms and Architectures: Technologies for faster, smaller models and more accurate inference
We are hiring! Feel free to reach out to me if you are interested to know more or apply here.

Jul 2017 - Present:

Lead Research Scientist at SNAP Research, Venice, CA

Jan 2017 - Apr 2017:

Research Intern at NVIDIA Research, Santa Clara, CA

Aug 2016 - Nov 2016:

Research Intern at Microsoft Research, Cambridge, UK

July 2010 - Sept 2012:

Team and Project Lead at HiQo-Solutions, Minsk, Belarus

June 2006 - June 2010:

Software Engineer at Todes Ltd, Minsk, Belarus

Nov 2012 - Apr 2017:

PhD Canditate at The Univesity of Trento, Italy

Sept 2014 - Feb 2015:

Research intern at the Robotics Institute, Carnegie-Mellon University

Sept 2009 - July 2010:

MSc at the Belarusian State University of Intormatics and Radioelectronics

Sept 2004 - July 2009:

B.Eng at the Belarusian State University of Intormatics and Radioelectronics

Teaching Computers to Imagine with Deep Generative Models

Sergey Tulyakov, Stéphane Lathuilière

University of Trento, Italy. November 19-26, 2019

Course material

[Oral] Transformable Bottleneck Networks

Kyle Olszewski, Sergey Tulyakov, Oliver Woodford, Hao Li, Linjie Luo

International Conference on Computer Vision, 2019

Paper Code Project

Laplace Landmark Localization

Joseph P Robinson, Yuncheng Li, Ning Zhang, Yun Fu, and Sergey Tulyakov

International Conference on Computer Vision, 2019

Paper

[Oral] Animating arbitrary objects via deep motion transfer

Aliaksandr Siarohin, Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci, Nicu Sebe

Computer Vision and Pattern Recognition, 2019

Paper Code Project

3D Guided Fine-Grained Face Manipulation

Zhenglin Geng, Chen Cao, Sergey Tulyakov

Computer Vision and Pattern Recognition, 2019

Paper

Real-Time Patch-Based Stylization of Portraits Using Generative Adversarial Network

David Futschik, Menglei Chai, Chen Cao, Chongyang Ma, Aleksei Stoliar, Sergey Korolev, Sergey Tulyakov, Michal Kučera, and Daniel Sýkora

ACM/EG Expressive Symposium, 2019

Paper Project

Train One Get One Free: Partially Supervised Neural Network for Bug Report Duplicate Detection and Clustering

Lahari Poddar, Leonardo Neves, William Brendel, Luis Marujo, Sergey Tulyakov, Pradeep Karuturi

North American Chapter of the Association for Computational Linguistics, 2019

Paper

MoCoGAN: Decomposing Motion and Content for Video Generation

Sergey Tulyakov, Ming-Yu Liu, Xiaodong Yang, Jan Kautz

Computer Vision and Pattern Recognition, 2018

Paper Code

Recurrent Convoltutional Shape Regression

Wei Wang, Sergey Tulyakov and Nicu Sebe

IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018

Paper

Hybrid VAE: Improving Deep Generative Models using Partial Observations

Sergey Tulyakov, Andrew Fitzgibbon, Sebastian Nowozin

Neural Information Processing Systems Workshops, 2017

Paper

Viewpoint-consistent 3D Face Alignment

Sergey Tulyakov, Laszlo A. Jeni, Jeffrey F. Cohn and Nicu Sebe

IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017

Paper

[Oral] Recurrent Convolutional Face Alignment

Wei Wang, Sergey Tulyakov and Nicu Sebe

Asian Conference on Computer Vision, 2016

Paper

[Oral] Self-Adaptive Matrix Completion for Heart Rate Estimation from Face Videos under Realistic Conditions

Sergey Tulyakov, Xavier Alameda Pineda, Elisa Ricci, Jijun Yin, Jeffrey Cohn and Nicu Sebe

Computer Vision and Pattern Recognition, 2016

Read more

Regressing a 3D Face Shape from a Single Image

Sergey Tulyakov and Nicu Sebe

International Conference on Computer Vision, 2015

Read more

FaceCept3D: Real Time 3D Face Tracking and Analysis

Sergey Tulyakov, Radu Vieriu, Enver Sangineto and Nicu Sebe

International Conference on Computer Vision Workshops, 2015

Paper

Facial Expression Recognition under a Wide Range of Head Poses

Radu Vieriu, Sergey Tulyakov, Stas Semeniuta, Enver Sangineto and Nicu Sebe

Automatic Face and Gesture Recognition, 2015

Code Paper

Robust Real-Time Extreme Head Pose Estimation

Sergey Tulyakov, Radu Vieriu, Stas Semeniuta and Nicu Sebe

International Conference on Pattern Recognition, 2014

Code Data Paper