Alert button

mPMR: A Multilingual Pre-trained Machine Reader at Scale

May 23, 2023
Weiwen Xu, Xin Li, Wai Lam, Lidong Bing

Figure 1 for mPMR: A Multilingual Pre-trained Machine Reader at Scale
Figure 2 for mPMR: A Multilingual Pre-trained Machine Reader at Scale
Figure 3 for mPMR: A Multilingual Pre-trained Machine Reader at Scale
Figure 4 for mPMR: A Multilingual Pre-trained Machine Reader at Scale

Share this with someone who'll enjoy it:

We present multilingual Pre-trained Machine Reader (mPMR), a novel method for multilingual machine reading comprehension (MRC)-style pre-training. mPMR aims to guide multilingual pre-trained language models (mPLMs) to perform natural language understanding (NLU) including both sequence classification and span extraction in multiple languages. To achieve cross-lingual generalization when only source-language fine-tuning data is available, existing mPLMs solely transfer NLU capability from a source language to target languages. In contrast, mPMR allows the direct inheritance of multilingual NLU capability from the MRC-style pre-training to downstream tasks. Therefore, mPMR acquires better NLU capability for target languages. mPMR also provides a unified solver for tackling cross-lingual span extraction and sequence classification, thereby enabling the extraction of rationales to explain the sentence-pair classification process.

* To appear at ACL 2023 main conference  
View paper onarxiv icon

Share this with someone who'll enjoy it: