Alert button

Domain-Adaptive Pretraining Methods for Dialogue Understanding

Add code
Alert button
May 28, 2021
Figure 1 for Domain-Adaptive Pretraining Methods for Dialogue Understanding

Share this with someone who'll enjoy it:

Language models like BERT and SpanBERT pretrained on open-domain data have obtained impressive gains on various NLP tasks. In this paper, we probe the effectiveness of domain-adaptive pretraining objectives on downstream tasks. In particular, three objectives, including a novel objective focusing on modeling predicate-argument relations, are evaluated on two challenging dialogue understanding tasks. Experimental results demonstrate that domain-adaptive pretraining with proper objectives can significantly improve the performance of a strong baseline on these tasks, achieving the new state-of-the-art performances.

* 6 pages, to appear in ACL2021  

Share this with someone who'll enjoy it: