Get our free extension to see links to code for papers anywhere online!

Chrome logo  Add to Chrome

Firefox logo Add to Firefox

Nebula-I: A General Framework for Collaboratively Training Deep Learning Models on Low-Bandwidth Cloud Clusters



Yang Xiang , Zhihua Wu , Weibao Gong , Siyu Ding , Xianjie Mo , Yuang Liu , Shuohuan Wang , Peng Liu , Yongshuai Hou , Long Li , Bin Wang , Shaohuai Shi , Yaqian Han , Yue Yu , Ge Li , Yu Sun , Yanjun Ma , Dianhai Yu

* 20 pages, 10 figures, technical report 

   Access Paper or Ask Questions

ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation



Shuohuan Wang , Yu Sun , Yang Xiang , Zhihua Wu , Siyu Ding , Weibao Gong , Shikun Feng , Junyuan Shang , Yanbin Zhao , Chao Pang , Jiaxiang Liu , Xuyi Chen , Yuxiang Lu , Weixin Liu , Xi Wang , Yangfan Bai , Qiuliang Chen , Li Zhao , Shiyong Li , Peng Sun , Dianhai Yu , Yanjun Ma , Hao Tian , Hua Wu , Tian Wu , Wei Zeng , Ge Li , Wen Gao , Haifeng Wang

* arXiv admin note: text overlap with arXiv:2107.02137 

   Access Paper or Ask Questions

ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation



Yu Sun , Shuohuan Wang , Shikun Feng , Siyu Ding , Chao Pang , Junyuan Shang , Jiaxiang Liu , Xuyi Chen , Yanbin Zhao , Yuxiang Lu , Weixin Liu , Zhihua Wu , Weibao Gong , Jianzhong Liang , Zhizhou Shang , Peng Sun , Wei Liu , Xuan Ouyang , Dianhai Yu , Hao Tian , Hua Wu , Haifeng Wang


   Access Paper or Ask Questions

ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora



Xuan Ouyang , Shuohuan Wang , Chao Pang , Yu Sun , Hao Tian , Hua Wu , Haifeng Wang


   Access Paper or Ask Questions

ERNIE-DOC: The Retrospective Long-Document Modeling Transformer



Siyu Ding , Junyuan Shang , Shuohuan Wang , Yu Sun , Hao Tian , Hua Wu , Haifeng Wang


   Access Paper or Ask Questions

Galileo at SemEval-2020 Task 12: Multi-lingual Learning for Offensive Language Identification using Pre-trained Language Models



Shuohuan Wang , Jiaxiang Liu , Xuan Ouyang , Yu Sun

* 8 pages, 2 figures, 6 tables. Accepted at Proceedings of 14th International Workshop on Semantic Evaluation (SemEval-2020) 

   Access Paper or Ask Questions

kk2018 at SemEval-2020 Task 9: Adversarial Training for Code-Mixing Sentiment Classification



Jiaxiang Liu , Xuyi Chen , Shikun Feng , Shuohuan Wang , Xuan Ouyang , Yu Sun , Zhengjie Huang , Weiyue Su


   Access Paper or Ask Questions

ERNIE at SemEval-2020 Task 10: Learning Word Emphasis Selection by Pre-trained Language Model



Zhengjie Huang , Shikun Feng , Weiyue Su , Xuyi Chen , Shuohuan Wang , Jiaxiang Liu , Xuan Ouyang , Yu Sun


   Access Paper or Ask Questions

ERNIE 2.0: A Continual Pre-training Framework for Language Understanding



Yu Sun , Shuohuan Wang , Yukun Li , Shikun Feng , Hao Tian , Hua Wu , Haifeng Wang

* 11 pages, 3 figures and 7 tables; 

   Access Paper or Ask Questions

1
2
>>