RayNet: A Simulation Platform for Developing Reinforcement Learning-Driven Network Protocols

Reinforcement Learning has gained significant momentum in the development of network protocols. However, learning-based protocols are still in their infancy, and substantial research is required to build deployable solutions. Developing a protocol based on reinforcement learning is a complex and challenging process that involves several model design decisions and requires significant training and evaluation in real or realistic network topologies. Network simulators offer RL-based protocols a highly effective training environment, because simulations are deterministic and can run in parallel. In this paper, we introduce RayNet, a scalable and adaptable simulation framework for the development of learning-based network protocols. RayNet integrates OMNeT++, a fully programmable network simulator, with Ray/RLlib, a scalable training platform for distributed reinforcement learning. RayNet facilitates the methodical development of RL-based network protocols with minimal overhead. We have developed a congestion control use case and present evidence that RayNet can be a valuable framework for the computer networks research community.