Adam Roberts

According to our database, Adam Roberts authored at least 55 papers between 2007 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of three.

2024

Scaling Instruction-Finetuned Language Models.

Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Y. Zhao, Yanping Huang, Andrew M. Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, Jason Wei

J. Mach. Learn. Res., 2024

Training LLMs over Neurally Compressed Text.

Brian Lester, Jaehoon Lee, Alex Alemi, Jeffrey Pennington, Adam Roberts, Jascha Sohl-Dickstein, Noah Constant

CoRR, 2024

Gemma: Open Models Based on Gemini Research and Technology.

Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, Pouya Tafti, Léonard Hussenot, Aakanksha Chowdhery, Adam Roberts, Aditya Barua, Alex Botev, Alex Castro-Ros, Ambrose Slone, Amélie Héliou, Andrea Tacchetti, Anna Bulanova, Antonia Paterson, Beth Tsai, Bobak Shahriari, Charline Le Lan, Christopher A. Choquette-Choo, Clément Crepy, Daniel Cer, Daphne Ippolito, David Reid, Elena Buchatskaya, Eric Ni, Eric Noland, Geng Yan, George Tucker, George-Cristian Muraru, Grigory Rozhdestvenskiy, Henryk Michalewski, Ian Tenney, Ivan Grishchenko, Jacob Austin, James Keeling, Jane Labanowski, Jean-Baptiste Lespiau, Jeff Stanway, Jenny Brennan, Jeremy Chen, Johan Ferret, Justin Chiu, et al.

CoRR, 2024

A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity.

Shayne Longpre, Gregory Yauney, Emily Reif, Katherine Lee, Adam Roberts, Barret Zoph, Denny Zhou, Jason Wei, Kevin Robinson, David Mimno, Daphne Ippolito

Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), 2024

Adaptive Accompaniment with ReaLchords.

Yusong Wu, Tim Cooijmans, Kyle Kastner, Adam Roberts, Ian Simon, Alexander Scarlatos, Chris Donahue, Cassie Tarakajian, Shayegan Omidshafiei, Aaron C. Courville, Pablo Samuel Castro, Natasha Jaques, Cheng-Zhi Anna Huang

Proceedings of the Forty-first International Conference on Machine Learning, 2024

2023

Scaling Up Models and Data with t5x and seqio.

Adam Roberts, Hyung Won Chung, Gaurav Mishra, Anselm Levskaya, James Bradbury, Daniel Andor, Sharan Narang, Brian Lester, Colin Gaffney, Afroz Mohiuddin, Curtis Hawthorne, Aitor Lewkowycz, Alex Salcianu, Marc van Zee, Jacob Austin, Sebastian Goodman, Livio Baldini Soares, Haitang Hu, Sasha Tsvyashchenko, Aakanksha Chowdhery, Jasmijn Bastings, Jannis Bulian, Xavier Garcia, Jianmo Ni, Andrew Chen, Kathleen Kenealy, Kehang Han, Michelle Casbon, Jonathan H. Clark, Stephan Lee, Dan Garrette, James Lee-Thorp, Colin Raffel, Noam Shazeer, Marvin Ritter, Maarten Bosma, Alexandre Passos, Jeremy Maitin-Shepard, Noah Fiedel, Mark Omernick, Brennan Saeta, Ryan Sepassi, Alexander Spiridonov, Joshua Newlan, Andrea Gesmundo

J. Mach. Learn. Res., 2023

PaLM: Scaling Language Modeling with Pathways.

Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern, Douglas Eck, Jeff Dean, Slav Petrov, Noah Fiedel

J. Mach. Learn. Res., 2023

Report of the 1st Workshop on Generative AI and Law.

A. Feder Cooper, Katherine Lee, James Grimmelmann, Daphne Ippolito, Christopher Callison-Burch, Christopher A. Choquette-Choo, Niloofar Mireshghallah, Miles Brundage, David Mimno, Madiha Zahrah Choksi, Jack M. Balkin, Nicholas Carlini, Christopher De Sa, Jonathan Frankle, Deep Ganguli, Bryant Gipson, Andres Guadamuz, Swee Leng Harris, Abigail Z. Jacobs, Elizabeth Joh, Gautam Kamath, Mark Lemley, Cass Matthews, Christine McLeavey, Corynne McSherry, Milad Nasr, Paul Ohm, Adam Roberts, Tom Rubin, Pamela Samuelson, Ludwig Schubert, Kristen Vaccaro, Luis Villa, Felix Wu, Elana Zeide

CoRR, 2023

SingSong: Generating musical accompaniments from singing.

Chris Donahue, Antoine Caillon, Adam Roberts, Ethan Manilow, Philippe Esling, Andrea Agostinelli, Mauro Verzetti, Ian Simon, Olivier Pietquin, Neil Zeghidour, Jesse H. Engel

CoRR, 2023

MusicLM: Generating Music From Text.

Andrea Agostinelli, Timo I. Denk, Zalán Borsos, Jesse H. Engel, Mauro Verzetti, Antoine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, Matthew Sharifi, Neil Zeghidour, Christian Havnø Frank

CoRR, 2023

The Flan Collection: Designing Data and Methods for Effective Instruction Tuning.

Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V. Le, Barret Zoph, Jason Wei, Adam Roberts

Proceedings of the International Conference on Machine Learning, 2023

Large Language Models Struggle to Learn Long-Tail Knowledge.

Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, Colin Raffel

Proceedings of the International Conference on Machine Learning, 2023

UniMax: Fairer and More Effective Language Sampling for Large-Scale Multilingual Pretraining.

Hyung Won Chung, Xavier Garcia, Adam Roberts, Yi Tay, Orhan Firat, Sharan Narang, Noah Constant

Proceedings of the Eleventh International Conference on Learning Representations, 2023

Crosslingual Generalization through Multitask Finetuning.

Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M. Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, Xiangru Tang, Dragomir Radev, Alham Fikri Aji, Khalid Almubarak, Samuel Albanie, Zaid Alyafeai, Albert Webson, Edward Raff, Colin Raffel

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023

Character-Aware Models Improve Visual Text Rendering.

Rosanne Liu, Dan Garrette, Chitwan Saharia, William Chan, Adam Roberts, Sharan Narang, Irina Blok, RJ Mical, Mohammad Norouzi, Noah Constant

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023

2022

ByT5: Towards a Token-Free Future with Pre-trained Byte-to-Byte Models.

Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, Colin Raffel

Trans. Assoc. Comput. Linguistics, 2022

VeLO: Training Versatile Learned Optimizers by Scaling Up.

Luke Metz, James Harrison, C. Daniel Freeman, Amil Merchant, Lucas Beyer, James Bradbury, Naman Agrawal, Ben Poole, Igor Mordatch, Adam Roberts, Jascha Sohl-Dickstein

CoRR, 2022

Scaling Instruction-Finetuned Language Models.

Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Y. Zhao, Yanping Huang, Andrew M. Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, Jason Wei

CoRR, 2022

What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?

Thomas Wang, Adam Roberts, Daniel Hesslow, Teven Le Scao, Hyung Won Chung, Iz Beltagy, Julien Launay, Colin Raffel

CoRR, 2022

Scaling Up Models and Data with t5x and seqio.

Adam Roberts, Hyung Won Chung, Anselm Levskaya, Gaurav Mishra, James Bradbury, Daniel Andor, Sharan Narang, Brian Lester, Colin Gaffney, Afroz Mohiuddin, Curtis Hawthorne, Aitor Lewkowycz, Alex Salcianu, Marc van Zee, Jacob Austin, Sebastian Goodman, Livio Baldini Soares, Haitang Hu, Sasha Tsvyashchenko, Aakanksha Chowdhery, Jasmijn Bastings, Jannis Bulian, Xavier Garcia, Jianmo Ni, Andrew Chen, Kathleen Kenealy, Jonathan H. Clark, Stephan Lee, Dan Garrette, James Lee-Thorp, Colin Raffel, Noam Shazeer, Marvin Ritter, Maarten Bosma, Alexandre Passos, Jeremy Maitin-Shepard, Noah Fiedel, Mark Omernick, Brennan Saeta, Ryan Sepassi, Alexander Spiridonov, Joshua Newlan, Andrea Gesmundo

CoRR, 2022

LaMDA: Language Models for Dialog Applications.

Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin, Dehao Chen, Yuanzhong Xu, Zhifeng Chen, Adam Roberts, Maarten Bosma, Yanqi Zhou, Chung-Ching Chang, Igor Krivokon, Will Rusch, Marc Pickett, Kathleen S. Meier-Hellstern, Meredith Ringel Morris, Tulsee Doshi, Renelito Delos Santos, Toju Duke, Johnny Soraker, Ben Zevenbergen, Vinodkumar Prabhakaran, Mark Diaz, Ben Hutchinson, Kristen Olson, Alejandra Molina, Erin Hoffman-John, Josh Lee, Lora Aroyo, Ravi Rajakumar, Alena Butryna, Matthew Lamm, Viktoriya Kuzmina, Joe Fenton, Aaron Cohen, Rachel Bernstein, Ray Kurzweil, Blaise Agüera y Arcas, Claire Cui, Marian Croak, Ed H. Chi, Quoc Le

CoRR, 2022

Multi-instrument Music Synthesis with Spectrogram Diffusion.

Curtis Hawthorne, Ian Simon, Adam Roberts, Neil Zeghidour, Josh Gardner, Ethan Manilow, Jesse H. Engel

Proceedings of the 23rd International Society for Music Information Retrieval Conference, 2022

What Language Model Architecture and Pretraining Objective Works Best for Zero-Shot Generalization?

Thomas Wang, Adam Roberts, Daniel Hesslow, Teven Le Scao, Hyung Won Chung, Iz Beltagy, Julien Launay, Colin Raffel

Proceedings of the International Conference on Machine Learning, 2022

2021

Extracting Training Data from Large Language Models.

Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom B. Brown, Dawn Song, Úlfar Erlingsson, Alina Oprea, Colin Raffel

Proceedings of the 30th USENIX Security Symposium, 2021

mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer.

Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, Colin Raffel

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021

Do Transformer Modifications Transfer Across Implementations and Applications?

Sharan Narang, Hyung Won Chung, Yi Tay, Liam Fedus, Thibault Févry, Michael Matena, Karishma Malkan, Noah Fiedel, Noam Shazeer, Zhenzhong Lan, Yanqi Zhou, Wei Li, Nan Ding, Jake Marcus, Adam Roberts, Colin Raffel

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021

2020

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu

J. Mach. Learn. Res., 2020

WT5?! Training Text-to-Text Models to Explain their Predictions.

Sharan Narang, Colin Raffel, Katherine Lee, Adam Roberts, Noah Fiedel, Karishma Malkan

CoRR, 2020

NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned.

Sewon Min, Jordan L. Boyd-Graber, Chris Alberti, Danqi Chen, Eunsol Choi, Michael Collins, Kelvin Guu, Hannaneh Hajishirzi, Kenton Lee, Jennimaria Palomaki, Colin Raffel, Adam Roberts, Tom Kwiatkowski, Patrick S. H. Lewis, Yuxiang Wu, Heinrich Küttler, Linqing Liu, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel, Sohee Yang, Minjoon Seo, Gautier Izacard, Fabio Petroni, Lucas Hosseini, Nicola De Cao, Edouard Grave, Ikuya Yamada, Sonse Shimaoka, Masatoshi Suzuki, Shumpei Miyawaki, Shun Sato, Ryo Takahashi, Jun Suzuki, Martin Fajcik, Martin Docekal, Karel Ondrej, Pavel Smrz, Hao Cheng, Yelong Shen, Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao, Barlas Oguz, Xilun Chen, Vladimir Karpukhin, Stan Peshterliev, Dmytro Okhonko, Michael Sejr Schlichtkrull, Sonal Gupta, Yashar Mehdad, Wen-tau Yih

Proceedings of the NeurIPS 2020 Competition and Demonstration Track, 2020

DDSP: Differentiable Digital Signal Processing.

Jesse H. Engel, Lamtharn Hantrakul, Chenjie Gu, Adam Roberts

Proceedings of the 8th International Conference on Learning Representations, 2020

How Much Knowledge Can You Pack Into the Parameters of a Language Model?

Adam Roberts, Colin Raffel, Noam Shazeer

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, 2020

2019

The Bach Doodle: Approachable music composition with machine learning at scale.

Cheng-Zhi Anna Huang, Curtis Hawthorne, Adam Roberts, Monica Dinculescu, James Wexler, Leon Hong, Jacob Howcroft

CoRR, 2019

Approachable Music Composition with Machine Learning at Scale.

Cheng-Zhi Anna Huang, Curtis Hawthorne, Adam Roberts, Monica Dinculescu, James Wexler, Leon Hong, Jacob Howcroft

Proceedings of the 20th International Society for Music Information Retrieval Conference, 2019

Fast and Flexible Neural Audio Synthesis.

Lamtharn Hantrakul, Jesse H. Engel, Adam Roberts, Chenjie Gu

Proceedings of the 20th International Society for Music Information Retrieval Conference, 2019

Learning to Groove with Inverse Sequence Transformations.

Jon Gillick, Adam Roberts, Jesse H. Engel, Douglas Eck, David Bamman

Proceedings of the 36th International Conference on Machine Learning, 2019

Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset.

Curtis Hawthorne, Andriy Stasyuk, Adam Roberts, Ian Simon, Cheng-Zhi Anna Huang, Sander Dieleman, Erich Elsen, Jesse H. Engel, Douglas Eck

Proceedings of the 7th International Conference on Learning Representations, 2019

GANSynth: Adversarial Neural Audio Synthesis.

Jesse H. Engel, Kumar Krishna Agrawal, Shuo Chen, Ishaan Gulrajani, Chris Donahue, Adam Roberts

Proceedings of the 7th International Conference on Learning Representations, 2019

Identifying the Intersections: User Experience + Research Scientist Collaboration in a Generative Machine Learning Interface.

Claire Kayacik, Sherol Chen, Signe Noerly, Jess Holbrook, Adam Roberts, Douglas Eck

Proceedings of the Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, 2019

2018

Learning a Latent Space of Multitrack Measures.

Ian Simon, Adam Roberts, Colin Raffel, Jesse H. Engel, Curtis Hawthorne, Douglas Eck

CoRR, 2018

Learning Latent Representations of Music to Generate Interactive Musical Palettes.

Adam Roberts, Jesse H. Engel, Sageev Oore, Douglas Eck

Proceedings of the Joint Proceedings of the ACM IUI 2018 Workshops co-located with the 23rd ACM Conference on Intelligent User Interfaces (ACM IUI 2018), 2018

Onsets and Frames: Dual-Objective Piano Transcription.

Curtis Hawthorne, Erich Elsen, Jialin Song, Adam Roberts, Ian Simon, Colin Raffel, Jesse H. Engel, Sageev Oore, Douglas Eck

Proceedings of the 19th International Society for Music Information Retrieval Conference, 2018

A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music.

Adam Roberts, Jesse H. Engel, Colin Raffel, Curtis Hawthorne, Douglas Eck

Proceedings of the 35th International Conference on Machine Learning, 2018

Latent Constraints: Learning to Generate Conditionally from Unconditional Generative Models.

Jesse H. Engel, Matthew D. Hoffman, Adam Roberts

Proceedings of the 6th International Conference on Learning Representations, 2018

2017

Counterpoint by Convolution.

Cheng-Zhi Anna Huang, Tim Cooijmans, Adam Roberts, Aaron C. Courville, Douglas Eck

Proceedings of the 18th International Society for Music Information Retrieval Conference, 2017

Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders.

Jesse H. Engel, Cinjon Resnick, Adam Roberts, Sander Dieleman, Mohammad Norouzi, Douglas Eck, Karen Simonyan

Proceedings of the 34th International Conference on Machine Learning, 2017

2015

Perception and representation of Bengali nasal vowels.

Sandra Kotzor, Adam Roberts, Allison Wetterlin, Aditi Lahiri

Proceedings of the 18th International Congress of Phonetic Sciences, 2015

2013

Ambiguous fragment assignment for high-throughput sequencing experiments.

Adam Roberts

PhD thesis, 2013

Fragment assignment in the cloud with eXpress-D.

Adam Roberts, Harvey Feng, Lior Pachter

BMC Bioinform., 2013

Updating RNA-Seq analyses after re-annotation.

Adam Roberts, Lorian Schaeffer, Lior Pachter

Bioinform., 2013

2012

Quantifying uniformity of mapped reads.

Valerie Hower, Richard Starfield, Adam Roberts, Lior Pachter

Bioinform., 2012

2011

Identification of novel transcripts in annotated genomes using RNA-Seq.

Adam Roberts, Harold Pimentel, Cole Trapnell, Lior Pachter

Bioinform., 2011

2009

Fast Statistical Alignment.

Robert K. Bradley, Adam Roberts, Michael E. Smoot, Sudeep Juvekar, Jaeyoung Do, Colin N. Dewey, Ian H. Holmes, Lior Pachter

PLoS Comput. Biol., 2009

Automated Risk Detection - What are the Key Elements Needed to Create a Multi-source, Pattern-based Risk Detection System?

Ian Blunt, Xavier Chitnis, Adam Roberts

Proceedings of the Second International Conference on Health Informatics, 2009

2007

Inferring missing genotypes in large SNP panels using fast nearest-neighbor searches over sliding windows.

Adam Roberts, Leonard McMillan, Wei Wang, Joel Parker, Ivan Rusyn, David Threadgill

Proceedings of the Proceedings 15th International Conference on Intelligent Systems for Molecular Biology (ISMB) & 6th European Conference on Computational Biology (ECCB), 2007

Sample Selection for Maximal Diversity.

Feng Pan, Adam Roberts, Leonard McMillan, David Threadgill, Wei Wang

Proceedings of the 7th IEEE International Conference on Data Mining (ICDM 2007), 2007