Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Ilya Sutskever

Unsupervised Neural Machine Translation with Generative Language Models Only


Oct 11, 2021
Jesse Michael Han, Igor Babuschkin, Harrison Edwards, Arvind Neelakantan, Tao Xu, Stanislas Polu, Alex Ray, Pranav Shyam, Aditya Ramesh, Alec Radford, Ilya Sutskever

* 10 pages 

  Access Paper or Ask Questions

Evaluating Large Language Models Trained on Code


Jul 14, 2021
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, Wojciech Zaremba

* corrected typos, added references, added authors, added acknowledgements 

  Access Paper or Ask Questions

Zero-Shot Text-to-Image Generation


Feb 26, 2021
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, Ilya Sutskever


  Access Paper or Ask Questions

Learning Transferable Visual Models From Natural Language Supervision


Feb 26, 2021
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, Ilya Sutskever


  Access Paper or Ask Questions

Generative Language Modeling for Automated Theorem Proving


Sep 07, 2020
Stanislas Polu, Ilya Sutskever

* 15+5 pages 

  Access Paper or Ask Questions

Language Models are Few-Shot Learners


Jun 05, 2020
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, Dario Amodei

* 40+32 pages 

  Access Paper or Ask Questions

Jukebox: A Generative Model for Music


Apr 30, 2020
Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever


  Access Paper or Ask Questions

Dota 2 with Large Scale Deep Reinforcement Learning


Dec 13, 2019
OpenAI, :, Christopher Berner, Greg Brockman, Brooke Chan, Vicki Cheung, Przemysław Dębiak, Christy Dennison, David Farhi, Quirin Fischer, Shariq Hashme, Chris Hesse, Rafal Józefowicz, Scott Gray, Catherine Olsson, Jakub Pachocki, Michael Petrov, Henrique Pondé de Oliveira Pinto, Jonathan Raiman, Tim Salimans, Jeremy Schlatter, Jonas Schneider, Szymon Sidor, Ilya Sutskever, Jie Tang, Filip Wolski, Susan Zhang


  Access Paper or Ask Questions

Deep Double Descent: Where Bigger Models and More Data Hurt


Dec 04, 2019
Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, Ilya Sutskever

* G.K. and Y.B. contributed equally 

  Access Paper or Ask Questions

Generating Long Sequences with Sparse Transformers


Apr 23, 2019
Rewon Child, Scott Gray, Alec Radford, Ilya Sutskever


  Access Paper or Ask Questions

FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models


Oct 22, 2018
Will Grathwohl, Ricky T. Q. Chen, Jesse Bettencourt, Ilya Sutskever, David Duvenaud

* 8 Pages, 6 figures 

  Access Paper or Ask Questions

GamePad: A Learning Environment for Theorem Proving


Jun 02, 2018
Daniel Huang, Prafulla Dhariwal, Dawn Song, Ilya Sutskever


  Access Paper or Ask Questions

Emergent Complexity via Multi-Agent Competition


Mar 14, 2018
Trapit Bansal, Jakub Pachocki, Szymon Sidor, Ilya Sutskever, Igor Mordatch

* Published as a conference paper at ICLR 2018 

  Access Paper or Ask Questions

Some Considerations on Learning to Explore via Meta-Reinforcement Learning


Mar 03, 2018
Bradly C. Stadie, Ge Yang, Rein Houthooft, Xi Chen, Yan Duan, Yuhuai Wu, Pieter Abbeel, Ilya Sutskever


  Access Paper or Ask Questions

Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments


Feb 23, 2018
Maruan Al-Shedivat, Trapit Bansal, Yuri Burda, Ilya Sutskever, Igor Mordatch, Pieter Abbeel

* Published as a conference paper at ICLR 2018 

  Access Paper or Ask Questions

One-Shot Imitation Learning


Dec 04, 2017
Yan Duan, Marcin Andrychowicz, Bradly C. Stadie, Jonathan Ho, Jonas Schneider, Ilya Sutskever, Pieter Abbeel, Wojciech Zaremba


  Access Paper or Ask Questions

Evolution Strategies as a Scalable Alternative to Reinforcement Learning


Sep 07, 2017
Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, Ilya Sutskever


  Access Paper or Ask Questions

An online sequence-to-sequence model for noisy speech recognition


Jun 16, 2017
Chung-Cheng Chiu, Dieterich Lawson, Yuping Luo, George Tucker, Kevin Swersky, Ilya Sutskever, Navdeep Jaitly

* arXiv admin note: substantial text overlap with arXiv:1608.01281 

  Access Paper or Ask Questions

Learning to Generate Reviews and Discovering Sentiment


Apr 06, 2017
Alec Radford, Rafal Jozefowicz, Ilya Sutskever


  Access Paper or Ask Questions

Third-Person Imitation Learning


Mar 06, 2017
Bradly C. Stadie, Pieter Abbeel, Ilya Sutskever


  Access Paper or Ask Questions

Variational Lossy Autoencoder


Mar 04, 2017
Xi Chen, Diederik P. Kingma, Tim Salimans, Yan Duan, Prafulla Dhariwal, John Schulman, Ilya Sutskever, Pieter Abbeel

* Added CIFAR10 experiments; ICLR 2017 

  Access Paper or Ask Questions

Improving Variational Inference with Inverse Autoregressive Flow


Jan 30, 2017
Diederik P. Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, Max Welling


  Access Paper or Ask Questions

RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning


Nov 10, 2016
Yan Duan, John Schulman, Xi Chen, Peter L. Bartlett, Ilya Sutskever, Pieter Abbeel

* 14 pages. Under review as a conference paper at ICLR 2017 

  Access Paper or Ask Questions

Extensions and Limitations of the Neural GPU


Nov 04, 2016
Eric Price, Wojciech Zaremba, Ilya Sutskever


  Access Paper or Ask Questions

A Neural Transducer


Aug 04, 2016
Navdeep Jaitly, David Sussillo, Quoc V. Le, Oriol Vinyals, Ilya Sutskever, Samy Bengio


  Access Paper or Ask Questions

Neural Programmer: Inducing Latent Programs with Gradient Descent


Aug 04, 2016
Arvind Neelakantan, Quoc V. Le, Ilya Sutskever

* Accepted as a conference paper at ICLR 2015 

  Access Paper or Ask Questions