Alert button

On Robust Numerical Solver for ODE via Self-Attention Mechanism

Feb 05, 2023
Zhongzhan Huang, Mingfu Liang, Liang Lin

Figure 1 for On Robust Numerical Solver for ODE via Self-Attention Mechanism
Figure 2 for On Robust Numerical Solver for ODE via Self-Attention Mechanism
Figure 3 for On Robust Numerical Solver for ODE via Self-Attention Mechanism
Figure 4 for On Robust Numerical Solver for ODE via Self-Attention Mechanism

Share this with someone who'll enjoy it:

With the development of deep learning techniques, AI-enhanced numerical solvers are expected to become a new paradigm for solving differential equations due to their versatility and effectiveness in alleviating the accuracy-speed trade-off in traditional numerical solvers. However, this paradigm still inevitably requires a large amount of high-quality data, whose acquisition is often very expensive in natural science and engineering problems. Therefore, in this paper, we explore training efficient and robust AI-enhanced numerical solvers with a small data size by mitigating intrinsic noise disturbances. We first analyze the ability of the self-attention mechanism to regulate noise in supervised learning and then propose a simple-yet-effective numerical solver, AttSolver, which introduces an additive self-attention mechanism to the numerical solution of differential equations based on the dynamical system perspective of the residual neural network. Our results on benchmarks, ranging from high-dimensional problems to chaotic systems, demonstrate the effectiveness of AttSolver in generally improving the performance of existing traditional numerical solvers without any elaborated model crafting. Finally, we analyze the convergence, generalization, and robustness of the proposed method experimentally and theoretically.

* Work in progress. Technical report  
View paper onarxiv icon

Share this with someone who'll enjoy it: