Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox


Can Monolingual Pretrained Models Help Cross-Lingual Classification?

Nov 10, 2019
Zewen Chi, Li Dong, Furu Wei, Xian-Ling Mao, Heyan Huang


Share this with someone who'll enjoy it:


Multilingual pretrained language models (such as multilingual BERT) have achieved impressive results for cross-lingual transfer. However, due to the constant model capacity, multilingual pre-training usually lags behind the monolingual competitors. In this work, we present two approaches to improve zero-shot cross-lingual classification, by transferring the knowledge from monolingual pretrained models to multilingual ones. Experimental results on two cross-lingual classification benchmarks show that our methods outperform vanilla multilingual fine-tuning.

* 5 pages 


   Access Paper Source



Share this with someone who'll enjoy it: