Recently deep generative models for static networks have been under active development and achieved significant success in application areas such as molecule design. However, many real-world problems involve temporal graphs whose topology and attribute values evolve dynamically over time, such as in the cases of protein folding, human mobility networks, and social network growth. However, deep generative models for temporal graphs has rarely been well explored yet and existing techniques for static graphs are not up to the task for temporal graphs since they cannot 1) encode and decode continuously-varying graph topology chronologically, 2) enforce validity via temporal constraints, and 3) ensure efficiency for information-lossless temporal resolution. To address these challenges, we propose a new model, called "Temporal Graph Generative Adversarial Network" (TG-GAN) for continuous-time temporal graph generation, by modeling the deep generative process for truncated temporal random walks and their compositions. Specifically, we first propose a novel temporal graph generator that jointly model truncated edge sequences, time budgets, and node attributes, with novel activation functions that enforce temporal validity constraints under recurrent architecture. In addition, a new temporal graph discriminator is proposed, which combines time and node encoding operations over a recurrent architecture to distinguish the generated sequences from the real ones sampled by a newly-developed truncated temporal random walk sampler. Extensive experiments on both synthetic and real-world datasets demonstrate TG-GAN significantly ourperforms the comparison methods in efficiency and effectiveness.