In quantitative finance, useful features are constructed by human experts. However, this method is of low efficient. Thus, automatic feature construction algorithms have received more and more attention. The SOTA technic in this field is to use reverse polish expression to represent the features, and then use genetic programming to reconstruct it. In this paper, we propose a new method, alpha discovery neural network, which can automatically construct features by using neural network. In this work, we made several contributions. Firstly, we put forward new object function by using empirical knowledge in financial signal processing, and we also fixed its undifferentiated problem. Secondly, we use model stealing technic to learn from other prior knowledge, which can bring enough diversity into our network. Thirdly, we come up with a method to measure the diversity of different financial features. Experiment shows that ADN can produce more diversified and higher informative features than GP. Besides, if we use GP's output to serve as prior knowledge, its final achievements will be significantly improved by using ADN.