Designing Universal Causal Deep Learning Models: The Case of Infinite-Dimensional Dynamical Systems from Stochastic Analysis

Deep learning (DL) is becoming indispensable to contemporary stochastic analysis and finance; nevertheless, it is still unclear how to design a principled DL framework for approximating infinite-dimensional causal operators. This paper proposes a "geometry-aware" solution to this open problem by introducing a DL model-design framework that takes a suitable infinite-dimensional linear metric spaces as inputs and returns a universal sequential DL models adapted to these linear geometries: we call these models Causal Neural Operators (CNO). Our main result states that the models produced by our framework can uniformly approximate on compact sets and across arbitrarily finite-time horizons H\"older or smooth trace class operators which causally map sequences between given linear metric spaces. Consequentially, we deduce that a single CNO can efficiently approximate the solution operator to a broad range of SDEs, thus allowing us to simultaneously approximate predictions from families of SDE models, which is vital to computational robust finance. We deduce that the CNO can approximate the solution operator to most stochastic filtering problems, implying that a single CNO can simultaneously filter a family of partially observed stochastic volatility models.

One-Shot Learning of Stochastic Differential Equations with Computational Graph Completion

We consider the problem of learning Stochastic Differential Equations of the form $dX_t = f(X_t)dt+\sigma(X_t)dW_t $ from one sample trajectory. This problem is more challenging than learning deterministic dynamical systems because one sample trajectory only provides indirect information on the unknown functions $f$, $\sigma$, and stochastic process $dW_t$ representing the drift, the diffusion, and the stochastic forcing terms, respectively. We propose a simple kernel-based solution to this problem that can be decomposed as follows: (1) Represent the time-increment map $X_t \rightarrow X_{t+dt}$ as a Computational Graph in which $f$, $\sigma$ and $dW_t$ appear as unknown functions and random variables. (2) Complete the graph (approximate unknown functions and random variables) via Maximum a Posteriori Estimation (given the data) with Gaussian Process (GP) priors on the unknown functions. (3) Learn the covariance functions (kernels) of the GP priors from data with randomized cross-validation. Numerical experiments illustrate the efficacy, robustness, and scope of our method.