Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox


Make Lead Bias in Your Favor: A Simple and Effective Method for News Summarization

Dec 25, 2019
Chenguang Zhu, Ziyi Yang, Robert Gmyr, Michael Zeng, Xuedong Huang


Share this with someone who'll enjoy it:


Lead bias is a common phenomenon in news summarization, where early parts of an article often contain the most salient information. While many algorithms exploit this fact in summary generation, it has a detrimental effect on teaching the model to discriminate and extract important information. We propose that the lead bias can be leveraged in a simple and effective way in our favor to pretrain abstractive news summarization models on large-scale unlabeled corpus: predicting the leading sentences using the rest of an article. Via careful data cleaning and filtering, our transformer-based pretrained model without any finetuning achieves remarkable results over various news summarization tasks. With further finetuning, our model outperforms many competitive baseline models. For example, the pretrained model without finetuning achieves state-of-the-art results on DUC-2003 and DUC-2004 datasets. The finetuned model obtains 3.2% higher ROUGE-1, 1.6% higher ROUGE-2 and 2.1% higher ROUGE-L scores than the best baseline model on XSum dataset. Human evaluations further show the effectiveness of our method.

* 7 pages, 2 figures 


   Access Paper Source



Share this with someone who'll enjoy it: