A prior-guided deep learning (DL) based interference mitigation approach is proposed for frequency modulated continuous wave (FMCW) radars. In this paper, the interference mitigation problem is tackled as a regression problem. Considering the complex-valued nature of radar signals, the complex-valued convolutional neural network is utilized as an architecture for implementation, which is different from the conventional real-valued counterparts. Meanwhile, as the useful beat signals of FMCW radars and interferences exhibit different distributions in the time-frequency domain, this prior feature is exploited as a regularization term to avoid overfitting of the learned representation. The effectiveness and accuracy of our proposed complex-valued fully convolutional network (CV-FCN) based interference mitigation approach are verified and analyzed through both simulated and measured radar signals. Compared to the real-valued counterparts, the CV-FCN shows a better interference mitigation performance with a potential of half memory reduction in low Signal to Interference plus Noise Ratio (SINR) scenarios. Moreover, the CV-FCN trained using only simulated data can be directly utilized for interference mitigation in various measured radar signals and shows a superior generalization capability. Furthermore, by incorporating the prior feature, the CV-FCN trained on only 1/8 of the full data achieves comparable performance as that on the full dataset in low SINR scenarios, and the training procedure converges faster.