Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Rohan Anil

Efficiently Identifying Task Groupings for Multi-Task Learning


Sep 10, 2021
Christopher Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, Chelsea Finn


  Access Paper or Ask Questions

Large-Scale Differentially Private BERT


Aug 03, 2021
Rohan Anil, Badih Ghazi, Vineet Gupta, Ravi Kumar, Pasin Manurangsi

* 12 pages, 6 figures 

  Access Paper or Ask Questions

LocoProp: Enhancing BackProp via Local Loss Optimization


Jun 11, 2021
Ehsan Amid, Rohan Anil, Manfred K. Warmuth


  Access Paper or Ask Questions

Knowledge distillation: A good teacher is patient and consistent


Jun 09, 2021
Lucas Beyer, Xiaohua Zhai, Amélie Royer, Larisa Markeeva, Rohan Anil, Alexander Kolesnikov

* Lucas, Xiaohua, Am\'elie, Larisa, and Alex contributed equally 

  Access Paper or Ask Questions

A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizes


Feb 16, 2021
Zachary Nado, Justin M. Gilmer, Christopher J. Shallue, Rohan Anil, George E. Dahl


  Access Paper or Ask Questions

Measuring and Harnessing Transference in Multi-Task Learning


Oct 29, 2020
Christopher Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, Chelsea Finn


  Access Paper or Ask Questions

Stochastic Optimization with Laggard Data Pipelines


Oct 26, 2020
Naman Agarwal, Rohan Anil, Tomer Koren, Kunal Talwar, Cyril Zhang

* Published as a conference paper at NeurIPS 2020 

  Access Paper or Ask Questions

Disentangling Adaptive Gradient Methods from Learning Rates


Feb 26, 2020
Naman Agarwal, Rohan Anil, Elad Hazan, Tomer Koren, Cyril Zhang


  Access Paper or Ask Questions

Second Order Optimization Made Practical


Feb 20, 2020
Rohan Anil, Vineet Gupta, Tomer Koren, Kevin Regan, Yoram Singer

* 18 pages, 15 figures 

  Access Paper or Ask Questions

Robust Bi-Tempered Logistic Loss Based on Bregman Divergences


Jun 08, 2019
Ehsan Amid, Manfred K. Warmuth, Rohan Anil, Tomer Koren


  Access Paper or Ask Questions

Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling


Feb 21, 2019
Jonathan Shen, Patrick Nguyen, Yonghui Wu, Zhifeng Chen, Mia X. Chen, Ye Jia, Anjuli Kannan, Tara Sainath, Yuan Cao, Chung-Cheng Chiu, Yanzhang He, Jan Chorowski, Smit Hinsu, Stella Laurenzo, James Qin, Orhan Firat, Wolfgang Macherey, Suyog Gupta, Ankur Bapna, Shuyuan Zhang, Ruoming Pang, Ron J. Weiss, Rohit Prabhavalkar, Qiao Liang, Benoit Jacob, Bowen Liang, HyoukJoong Lee, Ciprian Chelba, Sébastien Jean, Bo Li, Melvin Johnson, Rohan Anil, Rajat Tibrewal, Xiaobing Liu, Akiko Eriguchi, Navdeep Jaitly, Naveen Ari, Colin Cherry, Parisa Haghani, Otavio Good, Youlong Cheng, Raziel Alvarez, Isaac Caswell, Wei-Ning Hsu, Zongheng Yang, Kuan-Chieh Wang, Ekaterina Gonina, Katrin Tomanek, Ben Vanik, Zelin Wu, Llion Jones, Mike Schuster, Yanping Huang, Dehao Chen, Kazuki Irie, George Foster, John Richardson, Klaus Macherey, Antoine Bruguier, Heiga Zen, Colin Raffel, Shankar Kumar, Kanishka Rao, David Rybach, Matthew Murray, Vijayaditya Peddinti, Maxim Krikun, Michiel A. U. Bacchiani, Thomas B. Jablin, Rob Suderman, Ian Williams, Benjamin Lee, Deepti Bhatia, Justin Carlson, Semih Yavuz, Yu Zhang, Ian McGraw, Max Galkin, Qi Ge, Golan Pundak, Chad Whipkey, Todd Wang, Uri Alon, Dmitry Lepikhin, Ye Tian, Sara Sabour, William Chan, Shubham Toshniwal, Baohua Liao, Michael Nirschl, Pat Rondon


  Access Paper or Ask Questions

Memory-Efficient Adaptive Optimization for Large-Scale Learning


Jan 30, 2019
Rohan Anil, Vineet Gupta, Tomer Koren, Yoram Singer


  Access Paper or Ask Questions

Large scale distributed neural network training through online distillation


Apr 09, 2018
Rohan Anil, Gabriel Pereyra, Alexandre Passos, Robert Ormandi, George E. Dahl, Geoffrey E. Hinton


  Access Paper or Ask Questions

Wide & Deep Learning for Recommender Systems


Jun 24, 2016
Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, Rohan Anil, Zakaria Haque, Lichan Hong, Vihan Jain, Xiaobing Liu, Hemal Shah


  Access Paper or Ask Questions