Extreme Theory of Functional Connections: A Physics-Informed Neural Network Method for Solving Parametric Differential Equations

In this work we present a novel, accurate, and robust physics-informed method for solving problems involving parametric differential equations (DEs) called the Extreme Theory of Functional Connections, or X-TFC. The proposed method is a synergy of two recently developed frameworks for solving problems involving parametric DEs, 1) the Theory of Functional Connections, TFC, and the Physics-Informed Neural Networks, PINN. Although this paper focuses on the solution of exact problems involving parametric DEs (i.e. problems where the modeling error is negligible) with known parameters, X-TFC can also be used for data-driven solutions and data-driven discovery of parametric DEs. In the proposed method, the latent solution of the parametric DEs is approximated by a TFC constrained expression that uses a Neural Network (NN) as the free-function. This approximate solution form always analytically satisfies the constraints of the DE, while maintaining a NN with unconstrained parameters, like the Deep-TFC method. X-TFC differs from PINN and Deep-TFC; whereas PINN and Deep-TFC use a deep-NN, X-TFC uses a single-layer NN, or more precisely, an Extreme Learning Machine, ELM. This choice is based on the properties of the ELM algorithm. In order to numerically validate the method, it was tested over a range of problems including the approximation of solutions to linear and non-linear ordinary DEs (ODEs), systems of ODEs (SODEs), and partial DEs (PDEs). Furthermore, a few of these problems are of interest in physics and engineering such as the Classic Emden-Fowler equation, the Radiative Transfer (RT) equation, and the Heat-Transfer (HT) equation. The results show that X-TFC achieves high accuracy with low computational time and thus it is comparable with the other state-of-the-art methods.

Theory of Connections Applied to Support Vector Machines to Solve Differential Equations

Differential equations are used as numerical models to describe physical phenomena throughout the field of engineering and science, including heat and fluid flow, structural bending, and systems dynamics. Although there are many other techniques for finding approximate solutions to these equations, this paper looks to compare the application of the Theory of Connections (ToC) with one based on Support Vector Machines (SVM). The ToC method uses a constrained expression (an expression that always satisfies the differential equation constraints), which transforms the process of solving a differential equation into an unconstrained problem, and is ultimately solved via least-squares. In addition to individual analysis, the two methods are merged into a new methodology, called constrained SMVs (CSVM), by incorporating the SVM method into the ToC framework to solve unconstrained problems. Numerical tests are conducted on three sample problems: one first order linear ODEs, one first order non-linear ODE, and one second order linear ODE. Using the SVM method as a benchmark, a speed comparison is made for all the problems by timing the training period, and an accuracy comparison is made using the maximum error and mean-squared error on the training and test sets. In general, ToC is shown to be slightly faster (by an order of magnitude or less) and more accurate (by multiple orders of magnitude) over the SVM and CSVM approaches.