Recent works have shown that generative data augmentation, where synthetic samples generated from deep generative models are used to augment the training dataset, benefit certain NLP tasks. In this work, we extend this approach to the task of dialogue state tracking for goal-oriented dialogues, in which the data naturally exhibits a hierarchical structure over utterances and related annotations. Deep generative data augmentation for dialogue state tracking requires the generative model to be aware of the hierarchically structured data. We propose Variational Hierarchical Dialog Autoencoder (VHDA) for modeling various aspects of goal-oriented dialogues, including linguistic and underlying annotation structures. Our experiments show that our model is able to generate realistic and novel samples that improve the robustness of state-of-the-art dialogue state trackers, ultimately improving their final dialogue state tracking performances on several datasets.