Iterative Forgetting: Online Data Stream Regression Using Database-Inspired Adaptive Granulation

Many modern systems, such as financial, transportation, and telecommunications systems, are time-sensitive in the sense that they demand low-latency predictions for real-time decision-making. Such systems often have to contend with continuous unbounded data streams as well as concept drift, which are challenging requirements that traditional regression techniques are unable to cater to. There exists a need to create novel data stream regression methods that can handle these scenarios. We present a database-inspired datastream regression model that (a) uses inspiration from R*-trees to create granules from incoming datastreams such that relevant information is retained, (b) iteratively forgets granules whose information is deemed to be outdated, thus maintaining a list of only recent, relevant granules, and (c) uses the recent data and granules to provide low-latency predictions. The R*-tree-inspired approach also makes the algorithm amenable to integration with database systems. Our experiments demonstrate that the ability of this method to discard data produces a significant order-of-magnitude improvement in latency and training time when evaluated against the most accurate state-of-the-art algorithms, while the R*-tree-inspired granulation technique provides competitively accurate predictions

Thermodynamics-inspired Macroscopic States of Bounded Swarms

The collective behavior of swarms is extremely difficult to estimate or predict, even when the local agent rules are known and simple. The presented work seeks to leverage the similarities between fluids and swarm systems to generate a thermodynamics-inspired characterization of the collective behavior of robotic swarms. While prior works have borrowed tools from fluid dynamics to design swarming behaviors, they have usually avoided the task of generating a fluids-inspired macroscopic state (or macrostate) description of the swarm. This work will bridge the gap by seeking to answer the following question: is it possible to generate a small set of thermodynamics-inspired macroscopic properties that may later be used to quantify all possible collective behaviors of swarm systems? In this paper, we present three macroscopic properties analogous to pressure, temperature, and density of a gas, to describe the behavior of a swarm that is governed by only attractive and repulsive agent interactions. These properties are made to satisfy an equation similar to the ideal gas law, and also generalized to satisfy the virial equation of state for real gases. Finally, we investigate how swarm specifications such as density and average agent velocity affect the system macrostate.

Reward-Sharing Relational Networks in Multi-Agent Reinforcement Learning as a Framework for Emergent Behavior

In this work, we integrate `social' interactions into the MARL setup through a user-defined relational network and examine the effects of agent-agent relations on the rise of emergent behaviors. Leveraging insights from sociology and neuroscience, our proposed framework models agent relationships using the notion of Reward-Sharing Relational Networks (RSRN), where network edge weights act as a measure of how much one agent is invested in the success of (or `cares about') another. We construct relational rewards as a function of the RSRN interaction weights to collectively train the multi-agent system via a multi-agent reinforcement learning algorithm. The performance of the system is tested for a 3-agent scenario with different relational network structures (e.g., self-interested, communitarian, and authoritarian networks). Our results indicate that reward-sharing relational networks can significantly influence learned behaviors. We posit that RSRN can act as a framework where different relational networks produce distinct emergent behaviors, often analogous to the intuited sociological understanding of such networks.