Recently, Meta-learning has been shown as a promising way to improve the ability of learning from few data for Computer Vision. However, previous Meta-learning approaches exposed below problems: 1) they ignored the importance of attention mechanism for Meta-learner, leading the Meta-learner to be interfered by unimportant information; 2) they ignored the importance of past knowledge which can help the Meta-learner accurately understand the input data and further express them into high representations, and they train the Meta-learner to solve few shot learning task directly on the few original input data instead of on the high representations; 3) they suffer from a problem which we named as task-over-fitting (TOF) problem, which is probably caused by that they are requested to solve few shot learning task based on the original high dimensional input data, and redundant input information leads themselves to be easier to suffer from TOF. In this paper, we rethink the Meta-learning algorithm and propose that the attention mechanism and the past knowledge are crucial for the Meta-learner, and the Meta-learner should well use its past knowledge and express the input data into high representations to solve few shot learning tasks. Moreover, the Meta-learning approach should be free from the TOF problem. Based on these arguments, we redesign the Meta-learning algorithm to solve these three aforementioned problems, and proposed three methods. Extensive experiments demonstrate the effectiveness of our designation and methods with state-of-the-art performances on several few shot learning benchmarks. The source code of our proposed methods will be released soon.