RayNet: A Simulation Platform for Developing Reinforcement Learning-Driven Network Protocols

Reinforcement Learning has gained significant momentum in the development of network protocols. However, learning-based protocols are still in their infancy, and substantial research is required to build deployable solutions. Developing a protocol based on reinforcement learning is a complex and challenging process that involves several model design decisions and requires significant training and evaluation in real or realistic network topologies. Network simulators offer RL-based protocols a highly effective training environment, because simulations are deterministic and can run in parallel. In this paper, we introduce RayNet, a scalable and adaptable simulation framework for the development of learning-based network protocols. RayNet integrates OMNeT++, a fully programmable network simulator, with Ray/RLlib, a scalable training platform for distributed reinforcement learning. RayNet facilitates the methodical development of RL-based network protocols with minimal overhead. We have developed a congestion control use case and present evidence that RayNet can be a valuable framework for the computer networks research community.

Inferring the time-varying functional connectivity of large-scale computer networks from emitted events

We consider the problem of inferring the functional connectivity of a large-scale computer network from sparse time series of events emitted by its nodes. We do so under the following three domain-specific constraints: (a) non-stationarity of the functional connectivity due to unknown temporal changes in the network, (b) sparsity of the time-series of events that limits the effectiveness of classical correlation-based analysis, and (c) lack of an explicit model describing how events propagate through the network. Under the assumption that the probability of two nodes being functionally connected correlates with the mean delay between their respective events, we develop an inference method whose output is an undirected weighted network where the weight of an edge between two nodes denotes the probability of these nodes being functionally connected. Using a combination of windowing and convolution to calculate at each time window a score quantifying the likelihood of a pair of nodes emitting events in quick succession, we develop a model of time-varying connectivity whose parameters are determined by maximising the model's predictive power from one time window to the next. To assess the effectiveness of our inference method, we construct synthetic data for which ground truth is available and use these data to benchmark our approach against three state-of-the-art inference methods. We conclude by discussing its application to data from a real-world large-scale computer network.