Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

mu-Forcing: Training Variational Recurrent Autoencoders for Text Generation

May 24, 2019
Dayiheng Liu, Xu Yang, Feng He, Yuanyuan Chen, Jiancheng Lv

Share this with someone who'll enjoy it:

It has been previously observed that training Variational Recurrent Autoencoders (VRAE) for text generation suffers from serious uninformative latent variables problem. The model would collapse into a plain language model that totally ignore the latent variables and can only generate repeating and dull samples. In this paper, we explore the reason behind this issue and propose an effective regularizer based approach to address it. The proposed method directly injects extra constraints on the posteriors of latent variables into the learning process of VRAE, which can flexibly and stably control the trade-off between the KL term and the reconstruction term, making the model learn dense and meaningful latent representations. The experimental results show that the proposed method outperforms several strong baselines and can make the model learn interpretable latent variables and generate diverse meaningful sentences. Furthermore, the proposed method can perform well without using other strategies, such as KL annealing.

* To appear in the ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP) 

   Access Paper Source

Share this with someone who'll enjoy it: