On Reinforcement Learning, Effect Handlers, and the State Monad

We study the algebraic effects and handlers as a way to support decision-making abstractions in functional programs, whereas a user can ask a learning algorithm to resolve choices without implementing the underlying selection mechanism, and give a feedback by way of rewards. Differently from some recently proposed approach to the problem based on the selection monad [Abadi and Plotkin, LICS 2021], we express the underlying intelligence as a reinforcement learning algorithm implemented as a set of handlers for some of these algebraic operations, including those for choices and rewards. We show how we can in practice use algebraic operations and handlers -- as available in the programming language EFF -- to clearly separate the learning algorithm from its environment, thus allowing for a good level of modularity. We then show how the host language can be taken as a lambda-calculus with handlers, this way showing what the essential linguistic features are. We conclude by hinting at how type and effect systems could ensure safety properties, at the same time pointing at some directions for further work.

Types for Parallel Complexity in the Pi-calculus

Type systems as a way to control or analyze programs have been largely studied in the context of functional programming languages. Some of those work allow to extract from a typing derivation for a program a complexity bound on this program. We present how to adapt this result for parallel complexity in the pi-calculus, as a model of concurrency and parallel communication. We study two notions of time complexity: the total computation time without parallelism (the work) and the computation time under maximal parallelism (the span). We define reduction relations in the pi-calculus to capture those two notions, and we present two type systems from which one can extract a complexity bound on a process. The type systems are inspired by input/output types and size types, with temporal information about communications.