Filtered Partial Differential Equations: a robust surrogate constraint in physics-informed deep learning framework

Embedding physical knowledge into neural network (NN) training has been a hot topic. However, when facing the complex real-world, most of the existing methods still strongly rely on the quantity and quality of observation data. Furthermore, the neural networks often struggle to converge when the solution to the real equation is very complex. Inspired by large eddy simulation in computational fluid dynamics, we propose an improved method based on filtering. We analyzed the causes of the difficulties in physics informed machine learning, and proposed a surrogate constraint (filtered PDE, FPDE in short) of the original physical equations to reduce the influence of noisy and sparse observation data. In the noise and sparsity experiment, the proposed FPDE models (which are optimized by FPDE constraints) have better robustness than the conventional PDE models. Experiments demonstrate that the FPDE model can obtain the same quality solution with 100% higher noise and 12% quantity of observation data of the baseline. Besides, two groups of real measurement data are used to show the FPDE improvements in real cases. The final results show that FPDE still gives more physically reasonable solutions when facing the incomplete equation problem and the extremely sparse and high-noise conditions. For combining real-world experiment data into physics-informed training, the proposed FPDE constraint is useful and performs well in two real-world experiments: modeling the blood velocity in vessels and cell migration in scratches.

Stabilizing the Maximal Entropy Moment Method for Rarefied Gas Dynamics at Single-Precision

Developing extended hydrodynamics equations valid for both dense and rarefied gases remains a great challenge. A systematical solution for this challenge is the moment method describing both dense and rarefied gas behaviors with moments of gas molecule velocity distributions. Among moment methods, the maximal entropy moment method (MEM) stands out for its well-posedness and stability, which utilizes velocity distributions with maximized entropy. However, finding such distributions requires solving an ill-conditioned and computation-demanding optimization problem. This problem causes numerical overflow and breakdown when the numerical precision is insufficient, especially for flows like high-speed shock waves. It also prevents modern GPUs from accelerating optimization with their enormous single floating-point precision computation power. This paper aims to stabilize MEM, making it practical for simulating very strong normal shock waves on modern GPUs at single precision. We propose the gauge transformations for MEM, making the optimization less ill-conditioned. We also tackle numerical overflow and breakdown by adopting the canonical form of distribution and Newton's modified optimization method. With these techniques, we achieved a single-precision GPU simulation of a Mach 10 shock wave with 35 moments MEM, surpassing the previous double-precision results of Mach 4. Moreover, we argued that over-refined spatial mesh degrades both the accuracy and stability of MEM. Overall, this paper makes the maximal entropy moment method practical for simulating very strong normal shock waves on modern GPUs at single-precision, with significant stability improvement compared to previous methods.

Graph Partner Neural Networks for Semi-Supervised Learning on Graphs

Graph Convolutional Networks (GCNs) are powerful for processing graph-structured data and have achieved state-of-the-art performance in several tasks such as node classification, link prediction, and graph classification. However, it is inevitable for deep GCNs to suffer from an over-smoothing issue that the representations of nodes will tend to be indistinguishable after repeated graph convolution operations. To address this problem, we propose the Graph Partner Neural Network (GPNN) which incorporates a de-parameterized GCN and a parameter-sharing MLP. We provide empirical and theoretical evidence to demonstrate the effectiveness of the proposed MLP partner on tackling over-smoothing while benefiting from appropriate smoothness. To further tackle over-smoothing and regulate the learning process, we introduce a well-designed consistency contrastive loss and KL divergence loss. Besides, we present a graph enhancement technique to improve the overall quality of edges in graphs. While most GCNs can work with shallow architecture only, GPNN can obtain better results through increasing model depth. Experiments on various node classification tasks have demonstrated the state-of-the-art performance of GPNN. Meanwhile, extensive ablation studies are conducted to investigate the contributions of each component in tackling over-smoothing and improving performance.

Data Driven Macroscopic Modeling across Knudsen Numbers for Rarefied Gas Dynamics and Application to Rayleigh Scattering

Macroscopic modeling of the gas dynamics across Knudsen numbers from dense gas region to rarefied gas region remains a great challenge. The reason is macroscopic models lack accurate constitutive relations valid across different Knudsen numbers. To address this problem, we proposed a Data-driven, KnUdsen number Adaptive Linear constitutive relation model named DUAL. The DUAL model is accurate across a range of Knudsen numbers, from dense to rarefied, through learning to adapt Knudsen number change from observed data. It is consistent with the Navier-Stokes equation under the hydrodynamic limit, by utilizing a constrained neural network. In addition, it naturally satisfies the second law of thermodynamics and is robust to noisy data. We test the DUAL model on the calculation of Rayleigh scattering spectra. The DUAL model gives accurate spectra for various Knudsen numbers and is superior to traditional perturbation and moment expansion methods.

Probing Negative Sampling Strategies to Learn GraphRepresentations via Unsupervised Contrastive Learning

Graph representation learning has long been an important yet challenging task for various real-world applications. However, their downstream tasks are mainly performed in the settings of supervised or semi-supervised learning. Inspired by recent advances in unsupervised contrastive learning, this paper is thus motivated to investigate how the node-wise contrastive learning could be performed. Particularly, we respectively resolve the class collision issue and the imbalanced negative data distribution issue. Extensive experiments are performed on three real-world datasets and the proposed approach achieves the SOTA model performance.

Heterogeneous Graph Convolutional Networks for Temporal Community Detection

The Graph Convolutional Networks (GCN) has demonstrated superior performance in representing graph data, especially homogeneous graphs. However, the real-world graph data is usually heterogeneous and evolves with time, e.g., Facebook and DBLP, which has seldom been studied. To cope with this issue, we propose a novel approach named temporal heterogeneous graph convolutional network (THGCN). THGCN first embeds both spatial information and node attribute information together. Then, it captures short-term evolutionary patterns from the aggregations of embedded graph signals through compression network. Meanwhile, the long-term evolutionary patterns of heterogeneous graph data are also modeled via a TCN temporal convolutional network. To the best of our knowledge, this is the first attempt to model temporal heterogeneous graph data with a focus on community discovery task.