Get our free extension to see links to code for papers anywhere online!

Chrome logo  Add to Chrome

Firefox logo Add to Firefox
Picture for Barun Patra

Barun Patra
Alert button

Language Is Not All You Need: Aligning Perception with Language Models


Mar 01, 2023
Shaohan Huang, Li Dong, Wenhui Wang, Yaru Hao, Saksham Singhal, Shuming Ma, Tengchao Lv, Lei Cui, Owais Khan Mohammed, Barun Patra, Qiang Liu, Kriti Aggarwal, Zewen Chi, Johan Bjorck, Vishrav Chaudhary, Subhojit Som, Xia Song, Furu Wei

Add code
Alert button

Figure 1 for Language Is Not All You Need: Aligning Perception with Language Models
Figure 2 for Language Is Not All You Need: Aligning Perception with Language Models
Figure 3 for Language Is Not All You Need: Aligning Perception with Language Models
Figure 4 for Language Is Not All You Need: Aligning Perception with Language Models

   Access Paper or Ask Questions
  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

A Length-Extrapolatable Transformer


Dec 20, 2022
Yutao Sun, Li Dong, Barun Patra, Shuming Ma, Shaohan Huang, Alon Benhaim, Vishrav Chaudhary, Xia Song, Furu Wei

Add code
Alert button

Figure 1 for A Length-Extrapolatable Transformer
Figure 2 for A Length-Extrapolatable Transformer
Figure 3 for A Length-Extrapolatable Transformer
Figure 4 for A Length-Extrapolatable Transformer
* 9 pages 
   Access Paper or Ask Questions
  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

TorchScale: Transformers at Scale


Nov 23, 2022
Shuming Ma, Hongyu Wang, Shaohan Huang, Wenhui Wang, Zewen Chi, Li Dong, Alon Benhaim, Barun Patra, Vishrav Chaudhary, Xia Song, Furu Wei

Add code
Alert button

Figure 1 for TorchScale: Transformers at Scale
Figure 2 for TorchScale: Transformers at Scale
Figure 3 for TorchScale: Transformers at Scale
Figure 4 for TorchScale: Transformers at Scale
* Work in progress 
   Access Paper or Ask Questions
  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

Beyond English-Centric Bitexts for Better Multilingual Language Representation Learning


Oct 26, 2022
Barun Patra, Saksham Singhal, Shaohan Huang, Zewen Chi, Li Dong, Furu Wei, Vishrav Chaudhary, Xia Song

Add code
Alert button

Figure 1 for Beyond English-Centric Bitexts for Better Multilingual Language Representation Learning
Figure 2 for Beyond English-Centric Bitexts for Better Multilingual Language Representation Learning
Figure 3 for Beyond English-Centric Bitexts for Better Multilingual Language Representation Learning
Figure 4 for Beyond English-Centric Bitexts for Better Multilingual Language Representation Learning
* Work in progress 
   Access Paper or Ask Questions
  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

Foundation Transformers


Oct 19, 2022
Hongyu Wang, Shuming Ma, Shaohan Huang, Li Dong, Wenhui Wang, Zhiliang Peng, Yu Wu, Payal Bajaj, Saksham Singhal, Alon Benhaim, Barun Patra, Zhun Liu, Vishrav Chaudhary, Xia Song, Furu Wei

Add code
Alert button

Figure 1 for Foundation Transformers
Figure 2 for Foundation Transformers
Figure 3 for Foundation Transformers
Figure 4 for Foundation Transformers
* Work in progress 
   Access Paper or Ask Questions
  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

Language Model Decoding as Likelihood-Utility Alignment


Oct 13, 2022
Martin Josifoski, Maxime Peyrard, Frano Rajic, Jiheng Wei, Debjit Paul, Valentin Hartmann, Barun Patra, Vishrav Chaudhary, Emre Kıcıman, Boi Faltings, Robert West

Add code
Alert button

Figure 1 for Language Model Decoding as Likelihood-Utility Alignment
Figure 2 for Language Model Decoding as Likelihood-Utility Alignment
Figure 3 for Language Model Decoding as Likelihood-Utility Alignment
Figure 4 for Language Model Decoding as Likelihood-Utility Alignment

   Access Paper or Ask Questions
  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

On the Representation Collapse of Sparse Mixture of Experts


Apr 20, 2022
Zewen Chi, Li Dong, Shaohan Huang, Damai Dai, Shuming Ma, Barun Patra, Saksham Singhal, Payal Bajaj, Xia Song, Furu Wei

Add code
Alert button

Figure 1 for On the Representation Collapse of Sparse Mixture of Experts
Figure 2 for On the Representation Collapse of Sparse Mixture of Experts
Figure 3 for On the Representation Collapse of Sparse Mixture of Experts
Figure 4 for On the Representation Collapse of Sparse Mixture of Experts

   Access Paper or Ask Questions
  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

On Efficiently Acquiring Annotations for Multilingual Models


Apr 03, 2022
Joel Ruben Antony Moniz, Barun Patra, Matthew R. Gormley

Add code
Alert button

Figure 1 for On Efficiently Acquiring Annotations for Multilingual Models
Figure 2 for On Efficiently Acquiring Annotations for Multilingual Models
Figure 3 for On Efficiently Acquiring Annotations for Multilingual Models
Figure 4 for On Efficiently Acquiring Annotations for Multilingual Models
* ACL 2022 (Short Paper) 
   Access Paper or Ask Questions
  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

Invariant Language Modeling


Oct 16, 2021
Maxime Peyrard, Sarvjeet Singh Ghotra, Martin Josifoski, Vidhan Agarwal, Barun Patra, Dean Carignan, Emre Kiciman, Robert West

Add code
Alert button

Figure 1 for Invariant Language Modeling
Figure 2 for Invariant Language Modeling
Figure 3 for Invariant Language Modeling
Figure 4 for Invariant Language Modeling

   Access Paper or Ask Questions
  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email
1
2
>>