Collecting dialogue state labels, slots and values, for learning dialogue state tracking (DST) models can be costly, especially with the wide application of dialogue systems in new-rising domains. In this paper, we focus on how to learn a DST model efficiently with limited labeled data. We design a prompt learning framework for few-shot DST, which consists of two main components: value-based prompt and inverse prompt mechanism. This framework aims to utilize the language understanding and generation ability of pre-trained language models (PLM). First, we design value-based prompt functions to probe the DST-related knowledge from PLM, which do not rely on the known ontology of slots. Further, an inverse prompt mechanism is utilized to self-check the "prompted" knowledge and help the PLM understand the essence of DST task further. Experiments show that our model can generate unseen slots and outperforms existing state-of-the-art few-shot methods. It indicates that DST-related knowledge can be probed from PLM and utilized to address low-resource DST efficiently with the help of prompt learning.