The Concept of Representation in ML: Beyond Plato and Aristotle

Representation is a central concept in modern machine learning, where it usually refers to internal encodings that support learning and generalization. As models scale and their capabilities become increasingly human-level, this representational language sometimes shifts from an engineering context into the more philosophically loaded domain of mental representation. We argue that this is the case for recent claims about the convergence of representational properties across different AI models. In particular, we assess the arguments developed in The Platonic Representation Hypothesis, according to which this convergence is driven by a unified structure of reality. We examine this claim by introducing arguments and ideas from debates about mental representation in the philosophy of mind. We argue that these philosophical resources can clarify what is at stake in such claims, explain why alignment evidence alone is insufficient for strong metaphysical conclusions, and suggest directions for future research.

The Role of General Intelligence in Mathematical Reasoning

Objects are a centerpiece of the mathematical realm and our interaction with and reasoning about it, just as they are of the physical one (if not more). And humans' mathematical reasoning must ultimately be grounded in our general intelligence. Yet in contemporary cognitive science and A.I., the physical and mathematical domains are customarily explored separately, which allows for baking in assumptions for what objects are for the system - and missing potential connections. In this paper, I put the issue into its philosophical and cognitive context. I then describe an abstract theoretical framework for learning object representations, that makes room for mathematical objects on par with non-mathematical ones. Finally, I describe a case study that builds on that view to show how our general ability for integrating different aspects of objects effects our conception of the natural numbers.

Overcoming Forgetting in Federated Learning on Non-IID Data

We tackle the problem of Federated Learning in the non i.i.d. case, in which local models drift apart, inhibiting learning. Building on an analogy with Lifelong Learning, we adapt a solution for catastrophic forgetting to Federated Learning. We add a penalty term to the loss function, compelling all local models to converge to a shared optimum. We show that this can be done efficiently for communication (adding no further privacy risks), scaling with the number of nodes in the distributed setting. Our experiments show that this method is superior to competing ones for image recognition on the MNIST dataset.