Alert button

Few-Shot Learning of an Interleaved Text Summarization Model by Pretraining with Synthetic Data

Mar 08, 2021
Sanjeev Kumar Karn, Francine Chen, Yan-Ying Chen, Ulli Waltinger, Hinrich Schuetze

Figure 1 for Few-Shot Learning of an Interleaved Text Summarization Model by Pretraining with Synthetic Data
Figure 2 for Few-Shot Learning of an Interleaved Text Summarization Model by Pretraining with Synthetic Data
Figure 3 for Few-Shot Learning of an Interleaved Text Summarization Model by Pretraining with Synthetic Data
Figure 4 for Few-Shot Learning of an Interleaved Text Summarization Model by Pretraining with Synthetic Data

Share this with someone who'll enjoy it:

Interleaved texts, where posts belonging to different threads occur in a sequence, commonly occur in online chat posts, so that it can be time-consuming to quickly obtain an overview of the discussions. Existing systems first disentangle the posts by threads and then extract summaries from those threads. A major issue with such systems is error propagation from the disentanglement component. While end-to-end trainable summarization system could obviate explicit disentanglement, such systems require a large amount of labeled data. To address this, we propose to pretrain an end-to-end trainable hierarchical encoder-decoder system using synthetic interleaved texts. We show that by fine-tuning on a real-world meeting dataset (AMI), such a system out-performs a traditional two-step system by 22%. We also compare against transformer models and observed that pretraining with synthetic data both the encoder and decoder outperforms the BertSumExtAbs transformer model which pretrains only the encoder on a large dataset.

* Adapt-NLP: The Second Workshop on Domain Adaptation for NLP  
View paper onarxiv icon

Share this with someone who'll enjoy it: