Inverse Distance Weighted models (IDW) have been widely used for predicting and modeling multidimensional space in multimodal industrial processes. However, the more complex the structure of multidimensional space, the lower the performance of IDW models, and real industrial datasets tend to have more complex spatial structure. To solve this problem, a new framework for spatial prediction and modeling based on deep reinforcement learning network is proposed. In the proposed framework, the internal relationship between state and action is enhanced by reusing the state values in the Q network, and the convergence rate and stability of the deep reinforcement learning network are improved. The improved deep reinforcement learning network is then used to search for and learn the hyperparameters of each sample point in the inverse distance weighted model. These hyperparameters can reflect the spatial structure of the current industrial dataset to some extent. Then a spatial distribution of hyperparameters is constructed based on the learned hyperparameters. Each interpolation point obtains corresponding hyperparameters from the hyperparametric spatial distribution and brings them into the classical IDW models for prediction, thus achieving differential spatial prediction and modeling. The simulation results show that the proposed framework is suitable for real industrial datasets with complex spatial structure characteristics and is more accurate than current IDW models in spatial prediction.