Upper Generalization Bounds for Neural Oscillators

Neural oscillators that originate from the second-order ordinary differential equations (ODEs) have shown competitive performance in learning mappings between dynamic loads and responses of complex nonlinear structural systems. Despite this empirical success, theoretically quantifying the generalization capacities of their neural network architectures remains undeveloped. In this study, the neural oscillator consisting of a second-order ODE followed by a multilayer perceptron (MLP) is considered. Its upper probably approximately correct (PAC) generalization bound for approximating causal and uniformly continuous operators between continuous temporal function spaces and that for approximating the uniformly asymptotically incrementally stable second-order dynamical systems are derived by leveraging the Rademacher complexity framework. The theoretical results show that the estimation errors grow polynomially with respect to both the MLP size and the time length, thereby avoiding the curse of parametric complexity. Furthermore, the derived error bounds demonstrate that constraining the Lipschitz constants of the MLPs via loss function regularization can improve the generalization ability of the neural oscillator. A numerical study considering a Bouc-Wen nonlinear system under stochastic seismic excitation validates the theoretically predicted power laws of the estimation errors with respect to the sample size and time length, and confirms the effectiveness of constraining MLPs' matrix and vector norms in enhancing the performance of the neural oscillator under limited training data.

Navigating Uncertainties in Machine Learning for Structural Dynamics: A Comprehensive Review of Probabilistic and Non-Probabilistic Approaches in Forward and Inverse Problems

In the era of big data, machine learning (ML) has become a powerful tool in various fields, notably impacting structural dynamics. ML algorithms offer advantages by modeling physical phenomena based on data, even in the absence of underlying mechanisms. However, uncertainties such as measurement noise and modeling errors can compromise the reliability of ML predictions, highlighting the need for effective uncertainty awareness to enhance prediction robustness. This paper presents a comprehensive review on navigating uncertainties in ML, categorizing uncertainty-aware approaches into probabilistic methods (including Bayesian and frequentist perspectives) and non-probabilistic methods (such as interval learning and fuzzy learning). Bayesian neural networks, known for their uncertainty quantification and nonlinear mapping capabilities, are emphasized for their superior performance and potential. The review covers various techniques and methodologies for addressing uncertainties in ML, discussing fundamentals and implementation procedures of each method. While providing a concise overview of fundamental concepts, the paper refrains from in-depth critical explanations. Strengths and limitations of each approach are examined, along with their applications in structural dynamic forward problems like response prediction, sensitivity assessment, and reliability analysis, and inverse problems like system identification, model updating, and damage identification. Additionally, the review identifies research gaps and suggests future directions for investigations, aiming to provide comprehensive insights to the research community. By offering an extensive overview of both probabilistic and non-probabilistic approaches, this review aims to assist researchers and practitioners in making informed decisions when utilizing ML techniques to address uncertainties in structural dynamic problems.

Transport Map Coupling Filter for State-Parameter Estimation

Many dynamical systems are subjected to stochastic influences, such as random excitations, noise, and unmodeled behavior. Tracking the system's state and parameters based on a physical model is a common task for which filtering algorithms, such as Kalman filters and their non-linear extensions, are typically used. However, many of these filters use assumptions on the transition probabilities or the covariance model, which can lead to inaccuracies in non-linear systems. We will show the application of a stochastic coupling filter that can approximate arbitrary transition densities under non-Gaussian noise. The filter is based on transport maps, which couple the approximation densities to a user-chosen reference density, allowing for straightforward sampling and evaluation of probabilities.