Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox
Picture for Colin Raffel

Colin Raffel

Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP


Dec 20, 2021
Sabrina J. Mielke, Zaid Alyafeai, Elizabeth Salesky, Colin Raffel, Manan Dey, Matthias Gallé, Arun Raja, Chenglei Si, Wilson Y. Lee, Benoît Sagot, Samson Tan

Figure 1 for Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP
* 15 page preprint 
  Access Paper or Ask Questions

Training Neural Networks with Fixed Sparse Masks


Nov 18, 2021
Yi-Lin Sung, Varun Nair, Colin Raffel

Figure 1 for Training Neural Networks with Fixed Sparse Masks
Figure 2 for Training Neural Networks with Fixed Sparse Masks
Figure 3 for Training Neural Networks with Fixed Sparse Masks
Figure 4 for Training Neural Networks with Fixed Sparse Masks

  Access Paper or Ask Questions

Merging Models with Fisher-Weighted Averaging


Nov 18, 2021
Michael Matena, Colin Raffel

Figure 1 for Merging Models with Fisher-Weighted Averaging
Figure 2 for Merging Models with Fisher-Weighted Averaging
Figure 3 for Merging Models with Fisher-Weighted Averaging
Figure 4 for Merging Models with Fisher-Weighted Averaging

  Access Paper or Ask Questions

Multitask Prompted Training Enables Zero-Shot Task Generalization


Oct 15, 2021
Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Fevry, Jason Alan Fries, Ryan Teehan, Stella Biderman, Leo Gao, Tali Bers, Thomas Wolf, Alexander M. Rush

Figure 1 for Multitask Prompted Training Enables Zero-Shot Task Generalization
Figure 2 for Multitask Prompted Training Enables Zero-Shot Task Generalization
Figure 3 for Multitask Prompted Training Enables Zero-Shot Task Generalization
Figure 4 for Multitask Prompted Training Enables Zero-Shot Task Generalization
* https://github.com/bigscience-workshop/promptsource/ 
  Access Paper or Ask Questions

An Empirical Survey of Data Augmentation for Limited Data Learning in NLP


Jun 14, 2021
Jiaao Chen, Derek Tam, Colin Raffel, Mohit Bansal, Diyi Yang

Figure 1 for An Empirical Survey of Data Augmentation for Limited Data Learning in NLP
Figure 2 for An Empirical Survey of Data Augmentation for Limited Data Learning in NLP
Figure 3 for An Empirical Survey of Data Augmentation for Limited Data Learning in NLP
Figure 4 for An Empirical Survey of Data Augmentation for Limited Data Learning in NLP

  Access Paper or Ask Questions

On Training Sample Memorization: Lessons from Benchmarking Generative Modeling with a Large-scale Competition


Jun 06, 2021
Ching-Yuan Bai, Hsuan-Tien Lin, Colin Raffel, Wendy Chih-wen Kan

Figure 1 for On Training Sample Memorization: Lessons from Benchmarking Generative Modeling with a Large-scale Competition
Figure 2 for On Training Sample Memorization: Lessons from Benchmarking Generative Modeling with a Large-scale Competition
Figure 3 for On Training Sample Memorization: Lessons from Benchmarking Generative Modeling with a Large-scale Competition
Figure 4 for On Training Sample Memorization: Lessons from Benchmarking Generative Modeling with a Large-scale Competition
* In Proceedings of the 27th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), August 2021 
  Access Paper or Ask Questions

ByT5: Towards a token-free future with pre-trained byte-to-byte models


May 28, 2021
Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, Colin Raffel

Figure 1 for ByT5: Towards a token-free future with pre-trained byte-to-byte models
Figure 2 for ByT5: Towards a token-free future with pre-trained byte-to-byte models
Figure 3 for ByT5: Towards a token-free future with pre-trained byte-to-byte models
Figure 4 for ByT5: Towards a token-free future with pre-trained byte-to-byte models

  Access Paper or Ask Questions

Improving and Simplifying Pattern Exploiting Training


Mar 22, 2021
Derek Tam, Rakesh R Menon, Mohit Bansal, Shashank Srivastava, Colin Raffel

Figure 1 for Improving and Simplifying Pattern Exploiting Training
Figure 2 for Improving and Simplifying Pattern Exploiting Training
Figure 3 for Improving and Simplifying Pattern Exploiting Training
Figure 4 for Improving and Simplifying Pattern Exploiting Training
* 11 pages, 2 figures 
  Access Paper or Ask Questions

Do Transformer Modifications Transfer Across Implementations and Applications?


Feb 23, 2021
Sharan Narang, Hyung Won Chung, Yi Tay, William Fedus, Thibault Fevry, Michael Matena, Karishma Malkan, Noah Fiedel, Noam Shazeer, Zhenzhong Lan, Yanqi Zhou, Wei Li, Nan Ding, Jake Marcus, Adam Roberts, Colin Raffel

Figure 1 for Do Transformer Modifications Transfer Across Implementations and Applications?
Figure 2 for Do Transformer Modifications Transfer Across Implementations and Applications?
Figure 3 for Do Transformer Modifications Transfer Across Implementations and Applications?

  Access Paper or Ask Questions

NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned


Jan 01, 2021
Sewon Min, Jordan Boyd-Graber, Chris Alberti, Danqi Chen, Eunsol Choi, Michael Collins, Kelvin Guu, Hannaneh Hajishirzi, Kenton Lee, Jennimaria Palomaki, Colin Raffel, Adam Roberts, Tom Kwiatkowski, Patrick Lewis, Yuxiang Wu, Heinrich Küttler, Linqing Liu, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel, Sohee Yang, Minjoon Seo, Gautier Izacard, Fabio Petroni, Lucas Hosseini, Nicola De Cao, Edouard Grave, Ikuya Yamada, Sonse Shimaoka, Masatoshi Suzuki, Shumpei Miyawaki, Shun Sato, Ryo Takahashi, Jun Suzuki, Martin Fajcik, Martin Docekal, Karel Ondrej, Pavel Smrz, Hao Cheng, Yelong Shen, Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao, Barlas Oguz, Xilun Chen, Vladimir Karpukhin, Stan Peshterliev, Dmytro Okhonko, Michael Schlichtkrull, Sonal Gupta, Yashar Mehdad, Wen-tau Yih

Figure 1 for NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
Figure 2 for NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
Figure 3 for NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
Figure 4 for NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
* 26 pages 
  Access Paper or Ask Questions

Extracting Training Data from Large Language Models


Dec 14, 2020
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea, Colin Raffel

Figure 1 for Extracting Training Data from Large Language Models
Figure 2 for Extracting Training Data from Large Language Models
Figure 3 for Extracting Training Data from Large Language Models
Figure 4 for Extracting Training Data from Large Language Models

  Access Paper or Ask Questions

mT5: A massively multilingual pre-trained text-to-text transformer


Oct 23, 2020
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, Colin Raffel

Figure 1 for mT5: A massively multilingual pre-trained text-to-text transformer
Figure 2 for mT5: A massively multilingual pre-trained text-to-text transformer
Figure 3 for mT5: A massively multilingual pre-trained text-to-text transformer
Figure 4 for mT5: A massively multilingual pre-trained text-to-text transformer

  Access Paper or Ask Questions

WT5?! Training Text-to-Text Models to Explain their Predictions


Apr 30, 2020
Sharan Narang, Colin Raffel, Katherine Lee, Adam Roberts, Noah Fiedel, Karishma Malkan

Figure 1 for WT5?! Training Text-to-Text Models to Explain their Predictions
Figure 2 for WT5?! Training Text-to-Text Models to Explain their Predictions
Figure 3 for WT5?! Training Text-to-Text Models to Explain their Predictions
Figure 4 for WT5?! Training Text-to-Text Models to Explain their Predictions

  Access Paper or Ask Questions

How Much Knowledge Can You Pack Into the Parameters of a Language Model?


Feb 24, 2020
Adam Roberts, Colin Raffel, Noam Shazeer

Figure 1 for How Much Knowledge Can You Pack Into the Parameters of a Language Model?
Figure 2 for How Much Knowledge Can You Pack Into the Parameters of a Language Model?

  Access Paper or Ask Questions

Deflecting Adversarial Attacks


Feb 18, 2020
Yao Qin, Nicholas Frosst, Colin Raffel, Garrison Cottrell, Geoffrey Hinton

Figure 1 for Deflecting Adversarial Attacks
Figure 2 for Deflecting Adversarial Attacks
Figure 3 for Deflecting Adversarial Attacks
Figure 4 for Deflecting Adversarial Attacks

  Access Paper or Ask Questions

Top-K Training of GANs: Improving Generators by Making Critics Less Critical


Feb 14, 2020
Samarth Sinha, Anirudh Goyal, Colin Raffel, Augustus Odena

Figure 1 for Top-K Training of GANs: Improving Generators by Making Critics Less Critical
Figure 2 for Top-K Training of GANs: Improving Generators by Making Critics Less Critical
Figure 3 for Top-K Training of GANs: Improving Generators by Making Critics Less Critical
Figure 4 for Top-K Training of GANs: Improving Generators by Making Critics Less Critical

  Access Paper or Ask Questions

FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence


Jan 21, 2020
Kihyuk Sohn, David Berthelot, Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Han Zhang, Colin Raffel

Figure 1 for FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
Figure 2 for FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
Figure 3 for FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
Figure 4 for FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence

  Access Paper or Ask Questions

Towards GAN Benchmarks Which Require Generalization


Jan 10, 2020
Ishaan Gulrajani, Colin Raffel, Luke Metz

Figure 1 for Towards GAN Benchmarks Which Require Generalization
Figure 2 for Towards GAN Benchmarks Which Require Generalization
Figure 3 for Towards GAN Benchmarks Which Require Generalization
Figure 4 for Towards GAN Benchmarks Which Require Generalization
* ICLR 2019 conference paper 
  Access Paper or Ask Questions

ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring


Nov 21, 2019
David Berthelot, Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Kihyuk Sohn, Han Zhang, Colin Raffel

Figure 1 for ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring
Figure 2 for ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring
Figure 3 for ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring
Figure 4 for ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring

  Access Paper or Ask Questions

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer


Oct 24, 2019
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu

Figure 1 for Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Figure 2 for Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Figure 3 for Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Figure 4 for Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

  Access Paper or Ask Questions

Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions


Jul 05, 2019
Yao Qin, Nicholas Frosst, Sara Sabour, Colin Raffel, Garrison Cottrell, Geoffrey Hinton

Figure 1 for Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions
Figure 2 for Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions
Figure 3 for Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions
Figure 4 for Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions

  Access Paper or Ask Questions

Monotonic Infinite Lookback Attention for Simultaneous Machine Translation


Jun 12, 2019
Naveen Arivazhagan, Colin Cherry, Wolfgang Macherey, Chung-Cheng Chiu, Semih Yavuz, Ruoming Pang, Wei Li, Colin Raffel

Figure 1 for Monotonic Infinite Lookback Attention for Simultaneous Machine Translation
Figure 2 for Monotonic Infinite Lookback Attention for Simultaneous Machine Translation
* Accepted for publication at ACL 2019 
  Access Paper or Ask Questions

MixMatch: A Holistic Approach to Semi-Supervised Learning


May 06, 2019
David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, Colin Raffel

Figure 1 for MixMatch: A Holistic Approach to Semi-Supervised Learning
Figure 2 for MixMatch: A Holistic Approach to Semi-Supervised Learning
Figure 3 for MixMatch: A Holistic Approach to Semi-Supervised Learning
Figure 4 for MixMatch: A Holistic Approach to Semi-Supervised Learning

  Access Paper or Ask Questions

Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition


Mar 22, 2019
Yao Qin, Nicholas Carlini, Ian Goodfellow, Garrison Cottrell, Colin Raffel

Figure 1 for Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition
Figure 2 for Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition
Figure 3 for Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition

  Access Paper or Ask Questions

Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling


Feb 21, 2019
Jonathan Shen, Patrick Nguyen, Yonghui Wu, Zhifeng Chen, Mia X. Chen, Ye Jia, Anjuli Kannan, Tara Sainath, Yuan Cao, Chung-Cheng Chiu, Yanzhang He, Jan Chorowski, Smit Hinsu, Stella Laurenzo, James Qin, Orhan Firat, Wolfgang Macherey, Suyog Gupta, Ankur Bapna, Shuyuan Zhang, Ruoming Pang, Ron J. Weiss, Rohit Prabhavalkar, Qiao Liang, Benoit Jacob, Bowen Liang, HyoukJoong Lee, Ciprian Chelba, Sébastien Jean, Bo Li, Melvin Johnson, Rohan Anil, Rajat Tibrewal, Xiaobing Liu, Akiko Eriguchi, Navdeep Jaitly, Naveen Ari, Colin Cherry, Parisa Haghani, Otavio Good, Youlong Cheng, Raziel Alvarez, Isaac Caswell, Wei-Ning Hsu, Zongheng Yang, Kuan-Chieh Wang, Ekaterina Gonina, Katrin Tomanek, Ben Vanik, Zelin Wu, Llion Jones, Mike Schuster, Yanping Huang, Dehao Chen, Kazuki Irie, George Foster, John Richardson, Klaus Macherey, Antoine Bruguier, Heiga Zen, Colin Raffel, Shankar Kumar, Kanishka Rao, David Rybach, Matthew Murray, Vijayaditya Peddinti, Maxim Krikun, Michiel A. U. Bacchiani, Thomas B. Jablin, Rob Suderman, Ian Williams, Benjamin Lee, Deepti Bhatia, Justin Carlson, Semih Yavuz, Yu Zhang, Ian McGraw, Max Galkin, Qi Ge, Golan Pundak, Chad Whipkey, Todd Wang, Uri Alon, Dmitry Lepikhin, Ye Tian, Sara Sabour, William Chan, Shubham Toshniwal, Baohua Liao, Michael Nirschl, Pat Rondon

Figure 1 for Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
Figure 2 for Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
Figure 3 for Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling

  Access Paper or Ask Questions

Realistic Evaluation of Deep Semi-Supervised Learning Algorithms


Oct 26, 2018
Avital Oliver, Augustus Odena, Colin Raffel, Ekin D. Cubuk, Ian J. Goodfellow

Figure 1 for Realistic Evaluation of Deep Semi-Supervised Learning Algorithms
Figure 2 for Realistic Evaluation of Deep Semi-Supervised Learning Algorithms
Figure 3 for Realistic Evaluation of Deep Semi-Supervised Learning Algorithms
Figure 4 for Realistic Evaluation of Deep Semi-Supervised Learning Algorithms
* NIPS 2018 Proceedings 
  Access Paper or Ask Questions

A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music


Jul 30, 2018
Adam Roberts, Jesse Engel, Colin Raffel, Curtis Hawthorne, Douglas Eck

Figure 1 for A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music
Figure 2 for A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music
Figure 3 for A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music
Figure 4 for A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music
* ICML 2018 
 * ICML Camera Ready Version 
  Access Paper or Ask Questions

Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer


Jul 23, 2018
David Berthelot, Colin Raffel, Aurko Roy, Ian Goodfellow

Figure 1 for Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer
Figure 2 for Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer
Figure 3 for Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer
Figure 4 for Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer

  Access Paper or Ask Questions