Alert button

Constrained Text Generation with Global Guidance -- Case Study on CommonGen

Mar 12, 2021
Yixian Liu, Liwen Zhang, Wenjuan Han, Yue Zhang, Kewei Tu

Figure 1 for Constrained Text Generation with Global Guidance -- Case Study on CommonGen
Figure 2 for Constrained Text Generation with Global Guidance -- Case Study on CommonGen
Figure 3 for Constrained Text Generation with Global Guidance -- Case Study on CommonGen
Figure 4 for Constrained Text Generation with Global Guidance -- Case Study on CommonGen

Share this with someone who'll enjoy it:

This paper studies constrained text generation, which is to generate sentences under certain pre-conditions. We focus on CommonGen, the task of generating text based on a set of concepts, as a representative task of constrained text generation. Traditional methods mainly rely on supervised training to maximize the likelihood of target sentences.However, global constraints such as common sense and coverage cannot be incorporated into the likelihood objective of the autoregressive decoding process. In this paper, we consider using reinforcement learning to address the limitation, measuring global constraints including fluency, common sense and concept coverage with a comprehensive score, which serves as the reward for reinforcement learning. Besides, we design a guided decoding method at the word, fragment and sentence levels. Experiments demonstrate that our method significantly increases the concept coverage and outperforms existing models in various automatic evaluations.

View paper onarxiv icon

Share this with someone who'll enjoy it: