Physics-informed Neural Networks with Periodic Activation Functions for Solute Transport in Heterogeneous Porous Media

Solute transport in porous media is relevant to a wide range of applications in hydrogeology, geothermal energy, underground CO2 storage, and a variety of chemical engineering systems. Due to the complexity of solute transport in heterogeneous porous media, traditional solvers require high resolution meshing and are therefore expensive computationally. This study explores the application of a mesh-free method based on deep learning to accelerate the simulation of solute transport. We employ Physics-informed Neural Networks (PiNN) to solve solute transport problems in homogeneous and heterogeneous porous media governed by the advection-dispersion equation. Unlike traditional neural networks that learn from large training datasets, PiNNs only leverage the strong form mathematical models to simultaneously solve for multiple dependent or independent field variables (e.g., pressure and solute concentration fields). In this study, we construct PiNN using a periodic activation function to better represent the complex physical signals (i.e., pressure) and their derivatives (i.e., velocity). Several case studies are designed with the intention of investigating the proposed PiNN's capability to handle different degrees of complexity. A manual hyperparameter tuning method is used to find the best PiNN architecture for each test case. Point-wise error and mean square error (MSE) measures are employed to assess the performance of PiNNs' predictions against the ground truth solutions obtained analytically or numerically using the finite element method. Our findings show that the predictions of PiNN are in good agreement with the ground truth solutions while reducing computational complexity and cost by, at least, three orders of magnitude.

Physics-Guided, Physics-Informed, and Physics-Encoded Neural Networks in Scientific Computing

Recent breakthroughs in computing power have made it feasible to use machine learning and deep learning to advance scientific computing in many fields, such as fluid mechanics, solid mechanics, materials science, etc. Neural networks, in particular, play a central role in this hybridization. Due to their intrinsic architecture, conventional neural networks cannot be successfully trained and scoped when data is sparse; a scenario that is true in many scientific fields. Nonetheless, neural networks offer a strong foundation to digest physical-driven or knowledge-based constraints during training. Generally speaking, there are three distinct neural network frameworks to enforce underlying physics: (i) physics-guided neural networks (PgNN), (ii) physics-informed neural networks (PiNN) and (iii) physics-encoded neural networks (PeNN). These approaches offer unique advantages to accelerate the modeling of complex multiscale multi-physics phenomena. They also come with unique drawbacks and suffer from unresolved limitations (e.g., stability, convergence, and generalization) that call for further research. This study aims to present an in-depth review of the three neural network frameworks (i.e., PgNN, PiNN, and PeNN) used in scientific computing research. The state-of-the-art architectures and their applications are reviewed; limitations are discussed; and future research opportunities in terms of improving algorithms, considering causalities, expanding applications, and coupling scientific and deep learning solvers are presented. This critical review provides a solid starting point for researchers and engineers to comprehend how to integrate different layers of physics into neural networks.