In the design optimization of metal forming, it is increasingly significant to use surrogate models to analyse the finite element analysis (FEA) simulations. However, traditional surrogate models using scalar based machine learning methods (SBMLMs) fall in short of accuracy and generalizability. This is because SBMLMs fail to harness the location information of the simulations. To overcome these shortcomings, image based machine learning methods (IBMLMs) are leveraged in this paper. The underlying theory of location information, which supports the advantages of IBMLM, is qualitatively interpreted. Based on this theory, a Res-SE-U-Net IBMLM surrogate model is developed and compared with a multi-layer perceptron (MLP) as a referencing SBMLM surrogate model. It is demonstrated that the IBMLM model is advantageous over the MLP SBMLM model in accuracy, generalizability, robustness, and informativeness. This paper presents a promising methodology of leveraging IBMLMs in surrogate models to make maximum use of info from FEA results. Future prospective studies that inspired by this paper are also discussed.