CiRL: Open-Source Environments for Reinforcement Learning in Circular Economy and Net Zero

The demand of finite raw materials will keep increasing as they fuel modern society. Simultaneously, solutions for stopping carbon emissions in the short term are not available, thus making the net zero target extremely challenging to achieve at scale. The circular economy (CE) paradigm is gaining attention as a solution to address climate change and the uncertainties of supplies of critical materials. Hence, in this paper, we introduce CiRL, a deep reinforcement learning (DRL) library of environments focused on the circularity of both solid and fluid materials. The integration of DRL into the design of material circularity is possible thanks to the formalism of thermodynamical material networks, which is underpinned by compartmental dynamical thermodynamics. Along with the focus on circularity, this library has three more features: the new CE-oriented environments are in the state-space form, which is typically used in dynamical systems analysis and control designs; it is based on a state-of-the-art Python library of DRL algorithms, namely, Stable-Baselines3; and it is developed in Google Colaboratory to be accessible to researchers from different disciplines and backgrounds as is often the case for circular economy researchers and engineers. CiRL is publicly available.

A Unification Between Deep-Learning Vision, Compartmental Dynamical Thermodynamics, and Robotic Manipulation for a Circular Economy

The shift from a linear to a circular economy has the potential to simultaneously reduce uncertainties of material supplies and waste generation. To date, the development of robotic and, more generally, autonomous systems have been rarely integrated into circular economy implementation strategies. In this review, we merge deep-learning vision, compartmental dynamical thermodynamics, and robotic manipulation into a theoretically-coherent physics-based research framework to lay the foundations of circular flow designs of materials, and hence, to speed-up the transition from linearity to circularity. Then, we discuss opportunities for robotics in circular economy.