Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox
Picture for Colin Raffel

Colin Raffel

Improving and Simplifying Pattern Exploiting Training


Mar 22, 2021
Derek Tam, Rakesh R Menon, Mohit Bansal, Shashank Srivastava, Colin Raffel

Figure 1 for Improving and Simplifying Pattern Exploiting Training
Figure 2 for Improving and Simplifying Pattern Exploiting Training
Figure 3 for Improving and Simplifying Pattern Exploiting Training
Figure 4 for Improving and Simplifying Pattern Exploiting Training
* 11 pages, 2 figures 
  Access Paper or Ask Questions

Do Transformer Modifications Transfer Across Implementations and Applications?


Feb 23, 2021
Sharan Narang, Hyung Won Chung, Yi Tay, William Fedus, Thibault Fevry, Michael Matena, Karishma Malkan, Noah Fiedel, Noam Shazeer, Zhenzhong Lan, Yanqi Zhou, Wei Li, Nan Ding, Jake Marcus, Adam Roberts, Colin Raffel


  Access Paper or Ask Questions

NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned


Jan 01, 2021
Sewon Min, Jordan Boyd-Graber, Chris Alberti, Danqi Chen, Eunsol Choi, Michael Collins, Kelvin Guu, Hannaneh Hajishirzi, Kenton Lee, Jennimaria Palomaki, Colin Raffel, Adam Roberts, Tom Kwiatkowski, Patrick Lewis, Yuxiang Wu, Heinrich Küttler, Linqing Liu, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel, Sohee Yang, Minjoon Seo, Gautier Izacard, Fabio Petroni, Lucas Hosseini, Nicola De Cao, Edouard Grave, Ikuya Yamada, Sonse Shimaoka, Masatoshi Suzuki, Shumpei Miyawaki, Shun Sato, Ryo Takahashi, Jun Suzuki, Martin Fajcik, Martin Docekal, Karel Ondrej, Pavel Smrz, Hao Cheng, Yelong Shen, Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao, Barlas Oguz, Xilun Chen, Vladimir Karpukhin, Stan Peshterliev, Dmytro Okhonko, Michael Schlichtkrull, Sonal Gupta, Yashar Mehdad, Wen-tau Yih

Figure 1 for NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
Figure 2 for NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
Figure 3 for NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
Figure 4 for NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
* 26 pages 
  Access Paper or Ask Questions

Extracting Training Data from Large Language Models


Dec 14, 2020
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea, Colin Raffel

Figure 1 for Extracting Training Data from Large Language Models
Figure 2 for Extracting Training Data from Large Language Models
Figure 3 for Extracting Training Data from Large Language Models
Figure 4 for Extracting Training Data from Large Language Models

  Access Paper or Ask Questions

mT5: A massively multilingual pre-trained text-to-text transformer


Oct 23, 2020
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, Colin Raffel

Figure 1 for mT5: A massively multilingual pre-trained text-to-text transformer
Figure 2 for mT5: A massively multilingual pre-trained text-to-text transformer
Figure 3 for mT5: A massively multilingual pre-trained text-to-text transformer
Figure 4 for mT5: A massively multilingual pre-trained text-to-text transformer

  Access Paper or Ask Questions

WT5?! Training Text-to-Text Models to Explain their Predictions


Apr 30, 2020
Sharan Narang, Colin Raffel, Katherine Lee, Adam Roberts, Noah Fiedel, Karishma Malkan

Figure 1 for WT5?! Training Text-to-Text Models to Explain their Predictions
Figure 2 for WT5?! Training Text-to-Text Models to Explain their Predictions
Figure 3 for WT5?! Training Text-to-Text Models to Explain their Predictions
Figure 4 for WT5?! Training Text-to-Text Models to Explain their Predictions

  Access Paper or Ask Questions

How Much Knowledge Can You Pack Into the Parameters of a Language Model?


Feb 24, 2020
Adam Roberts, Colin Raffel, Noam Shazeer

Figure 1 for How Much Knowledge Can You Pack Into the Parameters of a Language Model?
Figure 2 for How Much Knowledge Can You Pack Into the Parameters of a Language Model?

  Access Paper or Ask Questions

Deflecting Adversarial Attacks


Feb 18, 2020
Yao Qin, Nicholas Frosst, Colin Raffel, Garrison Cottrell, Geoffrey Hinton

Figure 1 for Deflecting Adversarial Attacks
Figure 2 for Deflecting Adversarial Attacks
Figure 3 for Deflecting Adversarial Attacks
Figure 4 for Deflecting Adversarial Attacks

  Access Paper or Ask Questions

Top-K Training of GANs: Improving Generators by Making Critics Less Critical


Feb 14, 2020
Samarth Sinha, Anirudh Goyal, Colin Raffel, Augustus Odena

Figure 1 for Top-K Training of GANs: Improving Generators by Making Critics Less Critical
Figure 2 for Top-K Training of GANs: Improving Generators by Making Critics Less Critical
Figure 3 for Top-K Training of GANs: Improving Generators by Making Critics Less Critical
Figure 4 for Top-K Training of GANs: Improving Generators by Making Critics Less Critical

  Access Paper or Ask Questions

FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence


Jan 21, 2020
Kihyuk Sohn, David Berthelot, Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Han Zhang, Colin Raffel

Figure 1 for FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
Figure 2 for FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
Figure 3 for FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
Figure 4 for FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence

  Access Paper or Ask Questions

Towards GAN Benchmarks Which Require Generalization


Jan 10, 2020
Ishaan Gulrajani, Colin Raffel, Luke Metz

Figure 1 for Towards GAN Benchmarks Which Require Generalization
Figure 2 for Towards GAN Benchmarks Which Require Generalization
Figure 3 for Towards GAN Benchmarks Which Require Generalization
Figure 4 for Towards GAN Benchmarks Which Require Generalization
* ICLR 2019 conference paper 
  Access Paper or Ask Questions

ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring


Nov 21, 2019
David Berthelot, Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Kihyuk Sohn, Han Zhang, Colin Raffel

Figure 1 for ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring
Figure 2 for ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring
Figure 3 for ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring
Figure 4 for ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring

  Access Paper or Ask Questions

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer


Oct 24, 2019
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu

Figure 1 for Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Figure 2 for Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Figure 3 for Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Figure 4 for Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

  Access Paper or Ask Questions

Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions


Jul 05, 2019
Yao Qin, Nicholas Frosst, Sara Sabour, Colin Raffel, Garrison Cottrell, Geoffrey Hinton

Figure 1 for Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions
Figure 2 for Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions
Figure 3 for Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions
Figure 4 for Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions

  Access Paper or Ask Questions

Monotonic Infinite Lookback Attention for Simultaneous Machine Translation


Jun 12, 2019
Naveen Arivazhagan, Colin Cherry, Wolfgang Macherey, Chung-Cheng Chiu, Semih Yavuz, Ruoming Pang, Wei Li, Colin Raffel

Figure 1 for Monotonic Infinite Lookback Attention for Simultaneous Machine Translation
Figure 2 for Monotonic Infinite Lookback Attention for Simultaneous Machine Translation
* Accepted for publication at ACL 2019 
  Access Paper or Ask Questions

MixMatch: A Holistic Approach to Semi-Supervised Learning


May 06, 2019
David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, Colin Raffel

Figure 1 for MixMatch: A Holistic Approach to Semi-Supervised Learning
Figure 2 for MixMatch: A Holistic Approach to Semi-Supervised Learning
Figure 3 for MixMatch: A Holistic Approach to Semi-Supervised Learning
Figure 4 for MixMatch: A Holistic Approach to Semi-Supervised Learning

  Access Paper or Ask Questions

Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition


Mar 22, 2019
Yao Qin, Nicholas Carlini, Ian Goodfellow, Garrison Cottrell, Colin Raffel

Figure 1 for Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition
Figure 2 for Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition
Figure 3 for Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition

  Access Paper or Ask Questions

Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling


Feb 21, 2019
Jonathan Shen, Patrick Nguyen, Yonghui Wu, Zhifeng Chen, Mia X. Chen, Ye Jia, Anjuli Kannan, Tara Sainath, Yuan Cao, Chung-Cheng Chiu, Yanzhang He, Jan Chorowski, Smit Hinsu, Stella Laurenzo, James Qin, Orhan Firat, Wolfgang Macherey, Suyog Gupta, Ankur Bapna, Shuyuan Zhang, Ruoming Pang, Ron J. Weiss, Rohit Prabhavalkar, Qiao Liang, Benoit Jacob, Bowen Liang, HyoukJoong Lee, Ciprian Chelba, Sébastien Jean, Bo Li, Melvin Johnson, Rohan Anil, Rajat Tibrewal, Xiaobing Liu, Akiko Eriguchi, Navdeep Jaitly, Naveen Ari, Colin Cherry, Parisa Haghani, Otavio Good, Youlong Cheng, Raziel Alvarez, Isaac Caswell, Wei-Ning Hsu, Zongheng Yang, Kuan-Chieh Wang, Ekaterina Gonina, Katrin Tomanek, Ben Vanik, Zelin Wu, Llion Jones, Mike Schuster, Yanping Huang, Dehao Chen, Kazuki Irie, George Foster, John Richardson, Klaus Macherey, Antoine Bruguier, Heiga Zen, Colin Raffel, Shankar Kumar, Kanishka Rao, David Rybach, Matthew Murray, Vijayaditya Peddinti, Maxim Krikun, Michiel A. U. Bacchiani, Thomas B. Jablin, Rob Suderman, Ian Williams, Benjamin Lee, Deepti Bhatia, Justin Carlson, Semih Yavuz, Yu Zhang, Ian McGraw, Max Galkin, Qi Ge, Golan Pundak, Chad Whipkey, Todd Wang, Uri Alon, Dmitry Lepikhin, Ye Tian, Sara Sabour, William Chan, Shubham Toshniwal, Baohua Liao, Michael Nirschl, Pat Rondon

Figure 1 for Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
Figure 2 for Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
Figure 3 for Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling

  Access Paper or Ask Questions

Realistic Evaluation of Deep Semi-Supervised Learning Algorithms


Oct 26, 2018
Avital Oliver, Augustus Odena, Colin Raffel, Ekin D. Cubuk, Ian J. Goodfellow

Figure 1 for Realistic Evaluation of Deep Semi-Supervised Learning Algorithms
Figure 2 for Realistic Evaluation of Deep Semi-Supervised Learning Algorithms
Figure 3 for Realistic Evaluation of Deep Semi-Supervised Learning Algorithms
Figure 4 for Realistic Evaluation of Deep Semi-Supervised Learning Algorithms
* NIPS 2018 Proceedings 
  Access Paper or Ask Questions

A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music


Jul 30, 2018
Adam Roberts, Jesse Engel, Colin Raffel, Curtis Hawthorne, Douglas Eck

Figure 1 for A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music
Figure 2 for A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music
Figure 3 for A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music
Figure 4 for A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music
* ICML 2018 
 * ICML Camera Ready Version 
  Access Paper or Ask Questions

Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer


Jul 23, 2018
David Berthelot, Colin Raffel, Aurko Roy, Ian Goodfellow

Figure 1 for Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer
Figure 2 for Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer
Figure 3 for Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer
Figure 4 for Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer

  Access Paper or Ask Questions

Is Generator Conditioning Causally Related to GAN Performance?


Jun 19, 2018
Augustus Odena, Jacob Buckman, Catherine Olsson, Tom B. Brown, Christopher Olah, Colin Raffel, Ian Goodfellow

Figure 1 for Is Generator Conditioning Causally Related to GAN Performance?
Figure 2 for Is Generator Conditioning Causally Related to GAN Performance?
Figure 3 for Is Generator Conditioning Causally Related to GAN Performance?
Figure 4 for Is Generator Conditioning Causally Related to GAN Performance?

  Access Paper or Ask Questions

Onsets and Frames: Dual-Objective Piano Transcription


Jun 05, 2018
Curtis Hawthorne, Erich Elsen, Jialin Song, Adam Roberts, Ian Simon, Colin Raffel, Jesse Engel, Sageev Oore, Douglas Eck

Figure 1 for Onsets and Frames: Dual-Objective Piano Transcription
Figure 2 for Onsets and Frames: Dual-Objective Piano Transcription
Figure 3 for Onsets and Frames: Dual-Objective Piano Transcription
Figure 4 for Onsets and Frames: Dual-Objective Piano Transcription
* Examples available at https://goo.gl/magenta/onsets-frames-examples 
  Access Paper or Ask Questions

Learning a Latent Space of Multitrack Measures


Jun 01, 2018
Ian Simon, Adam Roberts, Colin Raffel, Jesse Engel, Curtis Hawthorne, Douglas Eck

Figure 1 for Learning a Latent Space of Multitrack Measures
Figure 2 for Learning a Latent Space of Multitrack Measures
Figure 3 for Learning a Latent Space of Multitrack Measures
Figure 4 for Learning a Latent Space of Multitrack Measures

  Access Paper or Ask Questions

Monotonic Chunkwise Attention


Feb 23, 2018
Chung-Cheng Chiu, Colin Raffel

Figure 1 for Monotonic Chunkwise Attention
Figure 2 for Monotonic Chunkwise Attention
Figure 3 for Monotonic Chunkwise Attention
Figure 4 for Monotonic Chunkwise Attention
* ICLR camera-ready version 
  Access Paper or Ask Questions

Learning Hard Alignments with Variational Inference


Nov 01, 2017
Dieterich Lawson, Chung-Cheng Chiu, George Tucker, Colin Raffel, Kevin Swersky, Navdeep Jaitly

Figure 1 for Learning Hard Alignments with Variational Inference
Figure 2 for Learning Hard Alignments with Variational Inference
Figure 3 for Learning Hard Alignments with Variational Inference

  Access Paper or Ask Questions

Online and Linear-Time Attention by Enforcing Monotonic Alignments


Jun 29, 2017
Colin Raffel, Minh-Thang Luong, Peter J. Liu, Ron J. Weiss, Douglas Eck

Figure 1 for Online and Linear-Time Attention by Enforcing Monotonic Alignments
Figure 2 for Online and Linear-Time Attention by Enforcing Monotonic Alignments
Figure 3 for Online and Linear-Time Attention by Enforcing Monotonic Alignments
Figure 4 for Online and Linear-Time Attention by Enforcing Monotonic Alignments
* ICML camera-ready version; 10 pages + 9 page appendix 
  Access Paper or Ask Questions

Training a Subsampling Mechanism in Expectation


Apr 08, 2017
Colin Raffel, Dieterich Lawson

Figure 1 for Training a Subsampling Mechanism in Expectation
Figure 2 for Training a Subsampling Mechanism in Expectation
Figure 3 for Training a Subsampling Mechanism in Expectation
Figure 4 for Training a Subsampling Mechanism in Expectation
* Camera-ready version. Includes additional figures in an appendix 
  Access Paper or Ask Questions