Symbolic Regression via Neural Networks

Identifying governing equations for a dynamical system is a topic of critical interest across an array of disciplines, from mathematics to engineering to biology. Machine learning -- specifically deep learning -- techniques have shown their capabilities in approximating dynamics from data, but a shortcoming of traditional deep learning is that there is little insight into the underlying mapping beyond its numerical output for a given input. This limits their utility in analysis beyond simple prediction. Simultaneously, a number of strategies exist which identify models based on a fixed dictionary of basis functions, but most either require some intuition or insight about the system, or are susceptible to overfitting or a lack of parsimony. Here we present a novel approach that combines the flexibility and accuracy of deep learning approaches with the utility of symbolic solutions: a deep neural network that generates a symbolic expression for the governing equations. We first describe the architecture for our model, then show the accuracy of our algorithm across a range of classical dynamical systems.

Expressive Symbolic Regression for Interpretable Models of Discrete-Time Dynamical Systems

Interpretable mathematical expressions defining discrete-time dynamical systems (iterated maps) can model many phenomena of scientific interest, enabling a deeper understanding of system behaviors. Since formulating governing expressions from first principles can be difficult, it is of particular interest to identify expressions for iterated maps given only their data streams. In this work, we consider a modified Symbolic Artificial Neural Network-Trained Expressions (SymANNTEx) architecture for this task, an architecture more expressive than others in the literature. We make a modification to the model pipeline to optimize the regression, then characterize the behavior of the adjusted model in identifying several classical chaotic maps. With the goal of parsimony, sparsity-inducing weight regularization and information theory-informed simplification are implemented. We show that our modified SymANNTEx model properly identifies single-state maps and achieves moderate success in approximating a dual-state attractor. These performances offer significant promise for data-driven scientific discovery and interpretation.

A Nonparametric Adaptive Nonlinear Statistical Filter

We use statistical learning methods to construct an adaptive state estimator for nonlinear stochastic systems. Optimal state estimation, in the form of a Kalman filter, requires knowledge of the system's process and measurement uncertainty. We propose that these uncertainties can be estimated from (conditioned on) past observed data, and without making any assumptions of the system's prior distribution. The system's prior distribution at each time step is constructed from an ensemble of least-squares estimates on sub-sampled sets of the data via jackknife sampling. As new data is acquired, the state estimates, process uncertainty, and measurement uncertainty are updated accordingly, as described in this manuscript.