Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Adam Roberts

ByT5: Towards a token-free future with pre-trained byte-to-byte models


May 28, 2021
Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, Colin Raffel


  Access Paper or Ask Questions

Do Transformer Modifications Transfer Across Implementations and Applications?


Feb 23, 2021
Sharan Narang, Hyung Won Chung, Yi Tay, William Fedus, Thibault Fevry, Michael Matena, Karishma Malkan, Noah Fiedel, Noam Shazeer, Zhenzhong Lan, Yanqi Zhou, Wei Li, Nan Ding, Jake Marcus, Adam Roberts, Colin Raffel


  Access Paper or Ask Questions

NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned


Jan 01, 2021
Sewon Min, Jordan Boyd-Graber, Chris Alberti, Danqi Chen, Eunsol Choi, Michael Collins, Kelvin Guu, Hannaneh Hajishirzi, Kenton Lee, Jennimaria Palomaki, Colin Raffel, Adam Roberts, Tom Kwiatkowski, Patrick Lewis, Yuxiang Wu, Heinrich Küttler, Linqing Liu, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel, Sohee Yang, Minjoon Seo, Gautier Izacard, Fabio Petroni, Lucas Hosseini, Nicola De Cao, Edouard Grave, Ikuya Yamada, Sonse Shimaoka, Masatoshi Suzuki, Shumpei Miyawaki, Shun Sato, Ryo Takahashi, Jun Suzuki, Martin Fajcik, Martin Docekal, Karel Ondrej, Pavel Smrz, Hao Cheng, Yelong Shen, Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao, Barlas Oguz, Xilun Chen, Vladimir Karpukhin, Stan Peshterliev, Dmytro Okhonko, Michael Schlichtkrull, Sonal Gupta, Yashar Mehdad, Wen-tau Yih

* 26 pages 

  Access Paper or Ask Questions

Extracting Training Data from Large Language Models


Dec 14, 2020
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea, Colin Raffel


  Access Paper or Ask Questions

mT5: A massively multilingual pre-trained text-to-text transformer


Oct 23, 2020
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, Colin Raffel


  Access Paper or Ask Questions

WT5?! Training Text-to-Text Models to Explain their Predictions


Apr 30, 2020
Sharan Narang, Colin Raffel, Katherine Lee, Adam Roberts, Noah Fiedel, Karishma Malkan


  Access Paper or Ask Questions

How Much Knowledge Can You Pack Into the Parameters of a Language Model?


Feb 24, 2020
Adam Roberts, Colin Raffel, Noam Shazeer


  Access Paper or Ask Questions

DDSP: Differentiable Digital Signal Processing


Jan 14, 2020
Jesse Engel, Lamtharn Hantrakul, Chenjie Gu, Adam Roberts


  Access Paper or Ask Questions

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer


Oct 24, 2019
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu


  Access Paper or Ask Questions

The Bach Doodle: Approachable music composition with machine learning at scale


Jul 14, 2019
Cheng-Zhi Anna Huang, Curtis Hawthorne, Adam Roberts, Monica Dinculescu, James Wexler, Leon Hong, Jacob Howcroft

* Proceedings of the 18th International Society for Music Information Retrieval Conference, ISMIR 2019 

  Access Paper or Ask Questions

Learning to Groove with Inverse Sequence Transformations


May 14, 2019
Jon Gillick, Adam Roberts, Jesse Engel, Douglas Eck, David Bamman

* ICML 2019, 11 pages, 6 figures 

  Access Paper or Ask Questions

GANSynth: Adversarial Neural Audio Synthesis


Apr 15, 2019
Jesse Engel, Kumar Krishna Agrawal, Shuo Chen, Ishaan Gulrajani, Chris Donahue, Adam Roberts

* Colab Notebook: http://goo.gl/magenta/gansynth-demo 

  Access Paper or Ask Questions

Counterpoint by Convolution


Mar 18, 2019
Cheng-Zhi Anna Huang, Tim Cooijmans, Adam Roberts, Aaron Courville, Douglas Eck

* Proceedings of the 18th International Society for Music Information Retrieval Conference, ISMIR 2017 

  Access Paper or Ask Questions

Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset


Oct 30, 2018
Curtis Hawthorne, Andriy Stasyuk, Adam Roberts, Ian Simon, Cheng-Zhi Anna Huang, Sander Dieleman, Erich Elsen, Jesse Engel, Douglas Eck

* Examples available at https://goo.gl/magenta/maestro-examples 

  Access Paper or Ask Questions

A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music


Jul 30, 2018
Adam Roberts, Jesse Engel, Colin Raffel, Curtis Hawthorne, Douglas Eck

* ICML 2018 
* ICML Camera Ready Version 

  Access Paper or Ask Questions

Onsets and Frames: Dual-Objective Piano Transcription


Jun 05, 2018
Curtis Hawthorne, Erich Elsen, Jialin Song, Adam Roberts, Ian Simon, Colin Raffel, Jesse Engel, Sageev Oore, Douglas Eck

* Examples available at https://goo.gl/magenta/onsets-frames-examples 

  Access Paper or Ask Questions

Learning a Latent Space of Multitrack Measures


Jun 01, 2018
Ian Simon, Adam Roberts, Colin Raffel, Jesse Engel, Curtis Hawthorne, Douglas Eck


  Access Paper or Ask Questions

Latent Constraints: Learning to Generate Conditionally from Unconditional Generative Models


Dec 21, 2017
Jesse Engel, Matthew Hoffman, Adam Roberts


  Access Paper or Ask Questions

Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders


Apr 05, 2017
Jesse Engel, Cinjon Resnick, Adam Roberts, Sander Dieleman, Douglas Eck, Karen Simonyan, Mohammad Norouzi


  Access Paper or Ask Questions