Ilya Sutskever

According to our database, Ilya Sutskever authored at least 82 papers between 2007 and 2024.

Collaborative distances:

• Dijkstra number of three.
• Erdős number of four.

2024

Scaling and evaluating sparse autoencoders.

Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, Jeffrey Wu

CoRR, 2024

Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision.

Collin Burns, Pavel Izmailov, Jan Hendrik Kirchner, Bowen Baker, Leo Gao, Leopold Aschenbrenner, Yining Chen, Adrien Ecoffet, Manas Joglekar, Jan Leike, Ilya Sutskever, Jeffrey Wu

Proceedings of the Forty-first International Conference on Machine Learning, 2024

Let's Verify Step by Step.

Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, Karl Cobbe

Proceedings of the Twelfth International Conference on Learning Representations, 2024

2023

Consistency Models.

Yang Song, Prafulla Dhariwal, Mark Chen, Ilya Sutskever

Proceedings of the International Conference on Machine Learning, 2023

Robust Speech Recognition via Large-Scale Weak Supervision.

Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, Ilya Sutskever

Proceedings of the International Conference on Machine Learning, 2023

Formal Mathematics Statement Curriculum Learning.

Stanislas Polu, Jesse Michael Han, Kunhao Zheng, Mantas Baksys, Igor Babuschkin, Ilya Sutskever

Proceedings of the Eleventh International Conference on Learning Representations, 2023

2022

GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models.

Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, Mark Chen

Proceedings of the International Conference on Machine Learning, 2022

2021

Unsupervised Neural Machine Translation with Generative Language Models Only.

Jesse Michael Han, Igor Babuschkin, Harrison Edwards, Arvind Neelakantan, Tao Xu, Stanislas Polu, Alex Ray, Pranav Shyam, Aditya Ramesh, Alec Radford, Ilya Sutskever

CoRR, 2021

Evaluating Large Language Models Trained on Code.

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondé de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Joshua Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, Wojciech Zaremba

CoRR, 2021

Zero-Shot Text-to-Image Generation.

Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, Ilya Sutskever

Proceedings of the 38th International Conference on Machine Learning, 2021

Learning Transferable Visual Models From Natural Language Supervision.

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, Ilya Sutskever

Proceedings of the 38th International Conference on Machine Learning, 2021

2020

Generative Language Modeling for Automated Theorem Proving.

Stanislas Polu, Ilya Sutskever

CoRR, 2020

Jukebox: A Generative Model for Music.

Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever

CoRR, 2020

Language Models are Few-Shot Learners.

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, Dario Amodei

Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, 2020

Distribution Augmentation for Generative Modeling.

Heewoo Jun, Rewon Child, Mark Chen, John Schulman, Aditya Ramesh, Alec Radford, Ilya Sutskever

Proceedings of the 37th International Conference on Machine Learning, 2020

Generative Pretraining From Pixels.

Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, Ilya Sutskever

Proceedings of the 37th International Conference on Machine Learning, 2020

Deep Double Descent: Where Bigger Models and More Data Hurt.

Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, Ilya Sutskever

Proceedings of the 8th International Conference on Learning Representations, 2020

2019

Dota 2 with Large Scale Deep Reinforcement Learning.

Christopher Berner, Greg Brockman, Brooke Chan, Vicki Cheung, Przemyslaw Debiak, Christy Dennison, David Farhi, Quirin Fischer, Shariq Hashme, Christopher Hesse, Rafal Józefowicz, Scott Gray, Catherine Olsson, Jakub Pachocki, Michael Petrov, Henrique Pondé de Oliveira Pinto, Jonathan Raiman, Tim Salimans, Jeremy Schlatter, Jonas Schneider, Szymon Sidor, Ilya Sutskever, Jie Tang, Filip Wolski, Susan Zhang

CoRR, 2019

Generating Long Sequences with Sparse Transformers.

Rewon Child, Scott Gray, Alec Radford, Ilya Sutskever

CoRR, 2019

GamePad: A Learning Environment for Theorem Proving.

Daniel Huang, Prafulla Dhariwal, Dawn Song, Ilya Sutskever

Proceedings of the 7th International Conference on Learning Representations, 2019

FFJORD: Free-Form Continuous Dynamics for Scalable Reversible Generative Models.

Will Grathwohl, Ricky T. Q. Chen, Jesse Bettencourt, Ilya Sutskever, David Duvenaud

Proceedings of the 7th International Conference on Learning Representations, 2019

2018

Some Considerations on Learning to Explore via Meta-Reinforcement Learning.

Bradly C. Stadie, Ge Yang, Rein Houthooft, Xi Chen, Yan Duan, Yuhuai Wu, Pieter Abbeel, Ilya Sutskever

CoRR, 2018

The Importance of Sampling inMeta-Reinforcement Learning.

Bradly C. Stadie, Ge Yang, Rein Houthooft, Xi Chen, Yan Duan, Yuhuai Wu, Pieter Abbeel, Ilya Sutskever

Proceedings of the Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, 2018

Emergent Complexity via Multi-Agent Competition.

Trapit Bansal, Jakub Pachocki, Szymon Sidor, Ilya Sutskever, Igor Mordatch

Proceedings of the 6th International Conference on Learning Representations, 2018

Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments.

Maruan Al-Shedivat, Trapit Bansal, Yura Burda, Ilya Sutskever, Igor Mordatch, Pieter Abbeel

Proceedings of the 6th International Conference on Learning Representations, 2018

2017

Evolution Strategies as a Scalable Alternative to Reinforcement Learning.

Tim Salimans, Jonathan Ho, Xi Chen, Ilya Sutskever

CoRR, 2017

Learning to Generate Reviews and Discovering Sentiment.

Alec Radford, Rafal Józefowicz, Ilya Sutskever

CoRR, 2017

An online sequence-to-sequence model for noisy speech recognition.

Chung-Cheng Chiu, Dieterich Lawson, Yuping Luo, George Tucker, Kevin Swersky, Ilya Sutskever, Navdeep Jaitly

CoRR, 2017

ImageNet classification with deep convolutional neural networks.

Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton

Commun. ACM, 2017

One-Shot Imitation Learning.

Yan Duan, Marcin Andrychowicz, Bradly C. Stadie, Jonathan Ho, Jonas Schneider, Ilya Sutskever, Pieter Abbeel, Wojciech Zaremba

Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 2017

Third Person Imitation Learning.

Bradly C. Stadie, Pieter Abbeel, Ilya Sutskever

Proceedings of the 5th International Conference on Learning Representations, 2017

Variational Lossy Autoencoder.

Xi Chen, Diederik P. Kingma, Tim Salimans, Yan Duan, Prafulla Dhariwal, John Schulman, Ilya Sutskever, Pieter Abbeel

Proceedings of the 5th International Conference on Learning Representations, 2017

Learning online alignments with continuous rewards policy gradient.

Yuping Luo, Chung-Cheng Chiu, Navdeep Jaitly, Ilya Sutskever

Proceedings of the 2017 IEEE International Conference on Acoustics, 2017

2016

Mastering the game of Go with deep neural networks and tree search.

David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Vedavyas Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy P. Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, Demis Hassabis

Nat., 2016

Neural Random Access Machines.

Karol Kurach, Marcin Andrychowicz, Ilya Sutskever

ERCIM News, 2016

Extensions and Limitations of the Neural GPU.

Eric Price, Wojciech Zaremba, Ilya Sutskever

CoRR, 2016

Neural Programmer: Inducing Latent Programs with Gradient Descent.

Arvind Neelakantan, Quoc V. Le, Ilya Sutskever

Proceedings of the 4th International Conference on Learning Representations, 2016

Multi-task Sequence to Sequence Learning.

Minh-Thang Luong, Quoc V. Le, Ilya Sutskever, Oriol Vinyals, Lukasz Kaiser

Proceedings of the 4th International Conference on Learning Representations, 2016

Neural GPUs Learn Algorithms.

Lukasz Kaiser, Ilya Sutskever

Proceedings of the 4th International Conference on Learning Representations, 2016

MuProp: Unbiased Backpropagation for Stochastic Neural Networks.

Shixiang Gu, Sergey Levine, Ilya Sutskever, Andriy Mnih

Proceedings of the 4th International Conference on Learning Representations, 2016

RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning.

Yan Duan, John Schulman, Xi Chen, Peter L. Bartlett, Ilya Sutskever, Pieter Abbeel

CoRR, 2016

TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems.

Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Gregory S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian J. Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Józefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Mané, Rajat Monga, Sherry Moore, Derek Gordon Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul A. Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda B. Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, Xiaoqiang Zheng

CoRR, 2016

Improving Variational Autoencoders with Inverse Autoregressive Flow.

Diederik P. Kingma, Tim Salimans, Rafal Józefowicz, Xi Chen, Ilya Sutskever, Max Welling

Proceedings of the Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, 2016

An Online Sequence-to-Sequence Model Using Partial Conditioning.

Navdeep Jaitly, Quoc V. Le, Oriol Vinyals, Ilya Sutskever, David Sussillo, Samy Bengio

Proceedings of the Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, 2016

InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets.

Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, Pieter Abbeel

Proceedings of the Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, 2016

Continuous Deep Q-Learning with Model-based Acceleration.

Shixiang Gu, Timothy P. Lillicrap, Ilya Sutskever, Sergey Levine

Proceedings of the 33nd International Conference on Machine Learning, 2016

2015

Reinforcement Learning Neural Turing Machines.

Wojciech Zaremba, Ilya Sutskever

CoRR, 2015

Towards Principled Unsupervised Learning.

Ilya Sutskever, Rafal Józefowicz, Karol Gregor, Danilo Jimenez Rezende, Timothy P. Lillicrap, Oriol Vinyals

CoRR, 2015

Adding Gradient Noise Improves Learning for Very Deep Networks.

Arvind Neelakantan, Luke Vilnis, Quoc V. Le, Ilya Sutskever, Lukasz Kaiser, Karol Kurach, James Martens

CoRR, 2015

Move Evaluation in Go Using Deep Convolutional Neural Networks.

Chris J. Maddison, Aja Huang, Ilya Sutskever, David Silver

Proceedings of the 3rd International Conference on Learning Representations, 2015

An Online Sequence-to-Sequence Model Using Partial Conditioning.

Navdeep Jaitly, Quoc V. Le, Oriol Vinyals, Ilya Sutskever, Samy Bengio

CoRR, 2015

Grammar as a Foreign Language.

Oriol Vinyals, Lukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, Geoffrey E. Hinton

Proceedings of the Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, 2015

An Empirical Exploration of Recurrent Network Architectures.

Rafal Józefowicz, Wojciech Zaremba, Ilya Sutskever

Proceedings of the 32nd International Conference on Machine Learning, 2015

Addressing the Rare Word Problem in Neural Machine Translation.

Thang Luong, Ilya Sutskever, Quoc V. Le, Oriol Vinyals, Wojciech Zaremba

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, 2015

2014

Dropout: a simple way to prevent neural networks from overfitting.

Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov

J. Mach. Learn. Res., 2014

Recurrent Neural Network Regularization.

Wojciech Zaremba, Ilya Sutskever, Oriol Vinyals

CoRR, 2014

Learning to Execute.

Wojciech Zaremba, Ilya Sutskever

CoRR, 2014

Intriguing properties of neural networks.

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, Rob Fergus

Proceedings of the 2nd International Conference on Learning Representations, 2014

Learning Factored Representations in a Deep Mixture of Experts.

David Eigen, Marc'Aurelio Ranzato, Ilya Sutskever

Proceedings of the 2nd International Conference on Learning Representations, 2014

Sequence to Sequence Learning with Neural Networks.

Ilya Sutskever, Oriol Vinyals, Quoc V. Le

Proceedings of the Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, 2014

2013

Training Recurrent Neural Networks.

Ilya Sutskever

PhD thesis, 2013

Exploiting Similarities among Languages for Machine Translation.

Tomás Mikolov, Quoc V. Le, Ilya Sutskever

CoRR, 2013

Distributed Representations of Words and Phrases and their Compositionality.

Tomás Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, Jeffrey Dean

Proceedings of the Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5-8, 2013

Stochastic k-Neighborhood Selection for Supervised and Unsupervised Learning.

Daniel Tarlow, Kevin Swersky, Laurent Charlin, Ilya Sutskever, Richard S. Zemel

Proceedings of the 30th International Conference on Machine Learning, 2013

On the importance of initialization and momentum in deep learning.

Ilya Sutskever, James Martens, George E. Dahl, Geoffrey E. Hinton

Proceedings of the 30th International Conference on Machine Learning, 2013

2012

Training Deep and Recurrent Networks with Hessian-Free Optimization.

James Martens, Ilya Sutskever

Proceedings of the Neural Networks: Tricks of the Trade - Second Edition, 2012

Improving neural networks by preventing co-adaptation of feature detectors

Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov

CoRR, 2012

Cardinality Restricted Boltzmann Machines.

Kevin Swersky, Daniel Tarlow, Ilya Sutskever, Ruslan Salakhutdinov, Richard S. Zemel, Ryan P. Adams

Proceedings of the Advances in Neural Information Processing Systems 25: 26th Annual Conference on Neural Information Processing Systems 2012. Proceedings of a meeting held December 3-6, 2012

Estimating the Hessian by Back-propagating Curvature.

James Martens, Ilya Sutskever, Kevin Swersky

Proceedings of the 29th International Conference on Machine Learning, 2012

2011

Generating Text with Recurrent Neural Networks.

Ilya Sutskever, James Martens, Geoffrey E. Hinton

Proceedings of the 28th International Conference on Machine Learning, 2011

Learning Recurrent Neural Networks with Hessian-Free Optimization.

James Martens, Ilya Sutskever

Proceedings of the 28th International Conference on Machine Learning, 2011

2010

Temporal-Kernel Recurrent Neural Networks.

Ilya Sutskever, Geoffrey E. Hinton

Neural Networks, 2010

On the Convergence Properties of Contrastive Divergence.

Ilya Sutskever, Tijmen Tieleman

Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, 2010

Parallelizable Sampling of Markov Random Fields.

James Martens, Ilya Sutskever

Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, 2010

2009

Modelling Relational Data using Bayesian Clustered Tensor Factorization.

Ilya Sutskever, Ruslan Salakhutdinov, Joshua B. Tenenbaum

Proceedings of the Advances in Neural Information Processing Systems 22: 23rd Annual Conference on Neural Information Processing Systems 2009. Proceedings of a meeting held 7-10 December 2009, 2009

A simpler unified analysis of budget perceptrons.

Ilya Sutskever

Proceedings of the 26th Annual International Conference on Machine Learning, 2009

2008

Deep, Narrow Sigmoid Belief Networks Are Universal Approximators.

Ilya Sutskever, Geoffrey E. Hinton

Neural Comput., 2008

The Recurrent Temporal Restricted Boltzmann Machine.

Ilya Sutskever, Geoffrey E. Hinton, Graham W. Taylor

Proceedings of the Advances in Neural Information Processing Systems 21, 2008

Using matrices to model symbolic relationship.

Ilya Sutskever, Geoffrey E. Hinton

Proceedings of the Advances in Neural Information Processing Systems 21, 2008

Mimicking Go Experts with Convolutional Neural Networks.

Ilya Sutskever, Vinod Nair

Proceedings of the Artificial Neural Networks, 2008

2007

Learning Multilevel Distributed Representations for High-Dimensional Sequences.

Ilya Sutskever, Geoffrey E. Hinton

Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics, 2007

Visualizing Similarity Data with a Mixture of Maps.

James Cook, Ilya Sutskever, Andriy Mnih, Geoffrey E. Hinton

Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics, 2007