Adaptive Frequency Green Light Optimal Speed Advisory based on Hybrid Actor-Critic Reinforcement Learning

Green Light Optimal Speed Advisory (GLOSA) system suggests speeds to vehicles to assist them in passing through intersections during green intervals, thus reducing traffic congestion and fuel consumption by minimizing the number of stops and idle times at intersections. However, previous research has focused on optimizing the GLOSA algorithm, neglecting the frequency of speed advisory by the GLOSA system. Specifically, some studies provide speed advisory profile at each decision step, resulting in redundant advisory, while others calculate the optimal speed for the vehicle only once, which cannot adapt to dynamic traffic. In this paper, we propose an Adaptive Frequency GLOSA (AF-GLOSA) model based on Hybrid Proximal Policy Optimization (H-PPO) method, which employs an actor-critic architecture with a hybrid actor network. The hybrid actor network consists of a discrete actor that outputs control gap and a continuous actor that outputs acceleration profiles. Additionally, we design a novel reward function that considers both travel efficiency and fuel consumption. The AF-GLOSA model is evaluated in comparison to traditional GLOSA and learning-based GLOSA methods in a three-lane intersection with a traffic signal in SUMO. The results demonstrate that the AF-GLOSA model performs best in reducing average stop times, fuel consumption and CO2 emissions.