Emotional support conversation (ESC) task can utilize various support strategies to help people relieve emotional distress and overcome the problem they face, which have attracted much attention in these years. The emotional support is a critical communication skill that should be trained into dialogue systems. Most existing studies predict support strategy according to current context and provide corresponding emotional support in response. However, these works ignore two significant characteristics of ESC. (a) Abundant prior knowledge exists in historical conversations, such as the responses to similar cases and the general order of support strategies, which has a great reference value for current conversation. (b) There is a one-to-many mapping relationship between context and support strategy, i.e.multiple strategies are reasonable for a single context. It lays a better foundation for the diversity of generations. To take into account these two key factors, we Prior Knowledge Enhanced emotional support conversation with latent variable model, PoKE. The proposed model fully taps the potential of prior knowledge in terms of exemplars and strategy sequence and then utilizes a latent variable to model the one-to-many relationship of support strategy. Furthermore, we introduce a memory schema to effectively incorporate encoded knowledge into decoder. Experiment results on benchmark dataset~(i.e., ESConv) show that our PoKE outperforms existing baselines on both automatic evaluation and human evaluation. Further experiments prove that abundant prior knowledge is conducive to high-quality emotional support, and a well-learned latent variable is critical to the diversity of generations.