Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Anirudh Ravula

DOCENT: Learning Self-Supervised Entity Representations from Large Document Collections


Feb 26, 2021
Yury Zemlyanskiy, Sudeep Gandhe, Ruining He, Bhargav Kanagal, Anirudh Ravula, Juraj Gottweis, Fei Sha, Ilya Eckstein

* To appear in the proceedings of EACL 2021 

  Access Paper or Ask Questions

RealFormer: Transformer Likes Residual Attention


Dec 23, 2020
Ruining He, Anirudh Ravula, Bhargav Kanagal, Joshua Ainslie

* 13 pages, 8 figures 

  Access Paper or Ask Questions

Informer: Transformer Likes Informed Attention


Dec 21, 2020
Ruining He, Anirudh Ravula, Bhargav Kanagal, Joshua Ainslie

* 13 pages, 8 figures 

  Access Paper or Ask Questions

Big Bird: Transformers for Longer Sequences


Jul 28, 2020
Manzil Zaheer, Guru Guruganesh, Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, Amr Ahmed


  Access Paper or Ask Questions

ETC: Encoding Long and Structured Data in Transformers


Apr 21, 2020
Joshua Ainslie, Santiago Ontanon, Chris Alberti, Philip Pham, Anirudh Ravula, Sumit Sanghai

* Updated ETC 512 results which mistakenly used a stale input format. Paper has not been peer-reviewed. An extended version will be submitted for review in the future 

  Access Paper or Ask Questions