Laws of Learning Dynamics and the Core of Learners

We formulate the fundamental laws governing learning dynamics, namely the conservation law and the decrease of total entropy. Within this framework, we introduce an entropy-based lifelong ensemble learning method. We evaluate its effectiveness by constructing an immunization mechanism to defend against transfer-based adversarial attacks on the CIFAR-10 dataset. Compared with a naive ensemble formed by simply averaging models specialized on clean and adversarial samples, the resulting logifold achieves higher accuracy in most test cases, with particularly large gains under strong perturbations.

Logifold: A Geometrical Foundation of Ensemble Machine Learning

We present a local-to-global and measure-theoretical approach to understanding datasets. The core idea is to formulate a logifold structure and to interpret network models with restricted domains as local charts of datasets. In particular, this provides a mathematical foundation for ensemble machine learning. Our experiments demonstrate that logifolds can be implemented to identify fuzzy domains and improve accuracy compared to taking average of model outputs. Additionally, we provide a theoretical example of a logifold, highlighting the importance of restricting to domains of classifiers in an ensemble.

A logifold structure on measure space

In this paper,we develop a local-to-global and measure-theoretical approach to understand datasets. The idea is to take network models with restricted domains as local charts of datasets. We develop the mathematical foundations for these structures, and show in experiments how it can be used to find fuzzy domains and to improve accuracy in data classification problems.