Neural Network Solutions to Differential Equations in Non-Convex Domains: Solving the Electric Field in the Slit-Well Microfluidic Device

The neural network method of solving differential equations is used to approximate the electric potential and corresponding electric field in the slit-well microfluidic device. The device's geometry is non-convex, making this a challenging problem to solve using the neural network method. To validate the method, the neural network solutions are compared to a reference solution obtained using the finite element method. Additional metrics are presented that measure how well the neural networks recover important physical invariants that are not explicitly enforced during training: spatial symmetries and conservation of electric flux. Finally, as an application-specific test of validity, neural network electric fields are incorporated into particle simulations. Conveniently, the same loss functional used to train the neural networks also seems to provide a reliable estimator of the networks' true errors, as measured by any of the metrics considered here. In all metrics, deep neural networks significantly outperform shallow neural networks, even when normalized by computational cost. Altogether, the results suggest that the neural network method can reliably produce solutions of acceptable accuracy for use in subsequent physical computations, such as particle simulations.

Neural Networks Trained to Solve Differential Equations Learn General Representations

We introduce a technique based on the singular vector canonical correlation analysis (SVCCA) for measuring the generality of neural network layers across a continuously-parametrized set of tasks. We illustrate this method by studying generality in neural networks trained to solve parametrized boundary value problems based on the Poisson partial differential equation. We find that the first hidden layer is general, and that deeper layers are successively more specific. Next, we validate our method against an existing technique that measures layer generality using transfer learning experiments. We find excellent agreement between the two methods, and note that our method is much faster, particularly for continuously-parametrized problems. Finally, we visualize the general representations of the first layers, and interpret them as generalized coordinates over the input domain.