Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox


Language Representation in Multilingual BERTand its applications to improve Cross-lingual Generalization

Oct 20, 2020
Chi-Liang Liu, Tsung-Yuan Hsu, Yung-Sung Chuang, Hung-yi Lee


Share this with someone who'll enjoy it:


A token embedding in multilingual BERT (m-BERT) contains both language and semantic information. We find that representation of a language can be obtained by simply averaging the embeddings of the tokens of the language. With the language representation, we can control the output languages of multilingual BERT by manipulating the token embeddings and achieve unsupervised token translation. We further propose a computationally cheap but effective approach to improve the cross-lingual ability of m-BERT based on the observation.



   Access Paper Source



Share this with someone who'll enjoy it: