A Letter on Convergence of In-Parameter-Linear Nonlinear Neural Architectures with Gradient Learnings

This letter summarizes and proves the concept of bounded-input bounded-state (BIBS) stability for weight convergence of a broad family of in-parameter-linear nonlinear neural architectures as it generally applies to a broad family of incremental gradient learning algorithms. A practical BIBS convergence condition results from the derived proofs for every individual learning point or batches for real-time applications.

An Approach to Stable Gradient Descent Adaptation of Higher-Order Neural Units

Stability evaluation of a weight-update system of higher-order neural units (HONUs) with polynomial aggregation of neural inputs (also known as classes of polynomial neural networks) for adaptation of both feedforward and recurrent HONUs by a gradient descent method is introduced. An essential core of the approach is based on spectral radius of a weight-update system, and it allows stability monitoring and its maintenance at every adaptation step individually. Assuring stability of the weight-update system (at every single adaptation step) naturally results in adaptation stability of the whole neural architecture that adapts to target data. As an aside, the used approach highlights the fact that the weight optimization of HONU is a linear problem, so the proposed approach can be generally extended to any neural architecture that is linear in its adaptable parameters.

Neural Network Approach to Railway Stand Lateral Skew Control

The paper presents a study of an adaptive approach to lateral skew control for an experimental railway stand. The preliminary experiments with the real experimental railway stand and simulations with its 3-D mechanical model, indicates difficulties of model-based control of the device. Thus, use of neural networks for identification and control of lateral skew shall be investigated. This paper focuses on real-data based modeling of the railway stand by various neural network models, i.e; linear neural unit and quadratic neural unit architectures. Furthermore, training methods of these neural architectures as such, real-time-recurrent-learning and a variation of back-propagation-through-time are examined, accompanied by a discussion of the produced experimental results.