Transfer Learning (TL) in Deep Neural Networks is gaining importance because in most of the cases, the labeling of data is costly and time-consuming. Additionally, TL provides effective weight initialization. This paper introduces the idea of Adaptive Transfer Learning in Deep Neural Networks for wind power prediction. Adaptive TL of Deep Neural Networks is proposed, which makes the proposed system an adaptive one as regards training on a different wind farm is concerned. The proposed technique is tested for short-term wind power predictions, where continuously arriving information has to be exploited. Adaptive TL not only helps in providing good weight initialization, but is also helpful to utilize the online data that is continuously being generated by wind farms. Additionally, the proposed technique is shown to transfer knowledge between different task domains (wind power to wind speed prediction) and from one region to another region. The simulation results show that proposed technique achieves average values of 0.0637,0.0986, and 0.0984 for the Mean-Absolute-Error, Root-Mean-Squared-Error, and Standard-Deviation-Error, respectively.