Alert button

Diffusion Models for Non-autoregressive Text Generation: A Survey

Mar 12, 2023
Yifan Li, Kun Zhou, Wayne Xin Zhao, Ji-Rong Wen

Figure 1 for Diffusion Models for Non-autoregressive Text Generation: A Survey
Figure 2 for Diffusion Models for Non-autoregressive Text Generation: A Survey

Share this with someone who'll enjoy it:

Non-autoregressive (NAR) text generation has attracted much attention in the field of natural language processing, which greatly reduces the inference latency but has to sacrifice the generation accuracy. Recently, diffusion models, a class of latent variable generative models, have been introduced into NAR text generation, showing improved generation quality. In this survey, we review the recent progress in diffusion models for NAR text generation. As the background, we first present the general definition of diffusion models and the text diffusion models, and then discuss their merits for NAR generation. As the core content, we further introduce two mainstream diffusion models in existing text diffusion works, and review the key designs of the diffusion process. Moreover, we discuss the utilization of pre-trained language models (PLMs) for text diffusion models and introduce optimization techniques for text data. Finally, we discuss several promising directions and conclude this paper. Our survey aims to provide researchers with a systematic reference of related research on text diffusion models for NAR generation.

* Work in progress  
View paper onarxiv icon

Share this with someone who'll enjoy it: