Learning Decentralized Traffic Signal Controllers with Multi-Agent Graph Reinforcement Learning

This paper considers optimal traffic signal control in smart cities, which has been taken as a complex networked system control problem. Given the interacting dynamics among traffic lights and road networks, attaining controller adaptivity and scalability stands out as a primary challenge. Capturing the spatial-temporal correlation among traffic lights under the framework of Multi-Agent Reinforcement Learning (MARL) is a promising solution. Nevertheless, existing MARL algorithms ignore effective information aggregation which is fundamental for improving the learning capacity of decentralized agents. In this paper, we design a new decentralized control architecture with improved environmental observability to capture the spatial-temporal correlation. Specifically, we first develop a topology-aware information aggregation strategy to extract correlation-related information from unstructured data gathered in the road network. Particularly, we transfer the road network topology into a graph shift operator by forming a diffusion process on the topology, which subsequently facilitates the construction of graph signals. A diffusion convolution module is developed, forming a new MARL algorithm, which endows agents with the capabilities of graph learning. Extensive experiments based on both synthetic and real-world datasets verify that our proposal outperforms existing decentralized algorithms.

Label-free Deep Learning Driven Secure Access Selection in Space-Air-Ground Integrated Networks

In Space-air-ground integrated networks (SAGIN), the inherent openness and extensive broadcast coverage expose these networks to significant eavesdropping threats. Considering the inherent co-channel interference due to spectrum sharing among multi-tier access networks in SAGIN, it can be leveraged to assist the physical layer security among heterogeneous transmissions. However, it is challenging to conduct a secrecy-oriented access strategy due to both heterogeneous resources and different eavesdropping models. In this paper, we explore secure access selection for a scenario involving multi-mode users capable of accessing satellites, unmanned aerial vehicles, or base stations in the presence of eavesdroppers. Particularly, we propose a Q-network approximation based deep learning approach for selecting the optimal access strategy for maximizing the sum secrecy rate. Meanwhile, the power optimization is also carried out by an unsupervised learning approach to improve the secrecy performance. Remarkably, two neural networks are trained by unsupervised learning and Q-network approximation which are both label-free methods without knowing the optimal solution as labels. Numerical results verify the efficiency of our proposed power optimization approach and access strategy, leading to enhanced secure transmission performance.

An Efficient and Reliable Asynchronous Federated Learning Scheme for Smart Public Transportation

Since the traffic conditions change over time, machine learning models that predict traffic flows must be updated continuously and efficiently in smart public transportation. Federated learning (FL) is a distributed machine learning scheme that allows buses to receive model updates without waiting for model training on the cloud. However, FL is vulnerable to poisoning or DDoS attacks since buses travel in public. Some work introduces blockchain to improve reliability, but the additional latency from the consensus process reduces the efficiency of FL. Asynchronous Federated Learning (AFL) is a scheme that reduces the latency of aggregation to improve efficiency, but the learning performance is unstable due to unreasonably weighted local models. To address the above challenges, this paper offers a blockchain-based asynchronous federated learning scheme with a dynamic scaling factor (DBAFL). Specifically, the novel committee-based consensus algorithm for blockchain improves reliability at the lowest possible cost of time. Meanwhile, the devised dynamic scaling factor allows AFL to assign reasonable weights to stale local models. Extensive experiments conducted on heterogeneous devices validate outperformed learning performance, efficiency, and reliability of DBAFL.

Green Interference Based Symbiotic Security in Integrated Satellite-terrestrial Communications

In this paper, we investigate secure transmissions in integrated satellite-terrestrial communications and the green interference based symbiotic security scheme is proposed. Particularly, the co-channel interference induced by the spectrum sharing between satellite and terrestrial networks and the inter-beam interference due to frequency reuse among satellite multi-beam serve as the green interference to assist the symbiotic secure transmission, where the secure transmissions of both satellite and terrestrial links are guaranteed simultaneously. Specifically, to realize the symbiotic security, we formulate a problem to maximize the sum secrecy rate of satellite users by cooperatively beamforming optimizing and a constraint of secrecy rate of each terrestrial user is guaranteed. Since the formulated problem is non-convex and intractable, the Taylor expansion and semi-definite relaxation (SDR) are adopted to further reformulate this problem, and the successive convex approximation (SCA) algorithm is designed to solve it. Finally, the tightness of the relaxation is proved. In addition, numerical results verify the efficiency of our proposed approach.

Collaborative Driving: Learning- Aided Joint Topology Formulation and Beamforming

Currently, autonomous vehicles are able to drive more naturally based on the driving policies learned from millions of driving miles in real environments. However, to further improve the automation level of vehicles is a challenging task, especially in the case of multi-vehicle cooperation. In recent heated discussions of 6G, millimeter-wave (mmWave) and terahertz (THz) bands are deemed to play important roles in new radio communication architectures and algorithms. To enable reliable autonomous driving in 6G, in this paper, we envision collaborative autonomous driving, a new framework that jointly controls driving topology and formulate vehicular networks in the mmWave/THz bands. As a swarm intelligence system, the collaborative driving scheme goes beyond existing autonomous driving patterns based on single-vehicle intelligence in terms of safety and efficiency. With efficient data sharing, the proposed framework is able to achieve cooperative sensing and load balancing so that improve sensing efficiency with saved computational resources. To deal with the new challenges in the collaborative driving framework, we further illustrate two promising approaches for mmWave/THz-based vehicle-to-vehicle (V2V) communications. Finally, we discuss several potential open research problems for the proposed collaborative driving scheme.

Learning Based Task Offloading in Digital Twin Empowered Internet of Vehicles

Mobile edge computing has become an effective and fundamental paradigm for futuristic autonomous vehicles to offload computing tasks. However, due to the high mobility of vehicles, the dynamics of the wireless conditions, and the uncertainty of the arrival computing tasks, it is difficult for a single vehicle to determine the optimal offloading strategy. In this paper, we propose a Digital Twin (DT) empowered task offloading framework for Internet of Vehicles. As a software agent residing in the cloud, a DT can obtain both global network information by using communications among DTs, and historical information of a vehicle by using the communications within the twin. The global network information and historical vehicular information can significantly facilitate the offloading. In specific, to preserve the precious computing resource at different levels for most appropriate computing tasks, we integrate a learning scheme based on the prediction of futuristic computing tasks in DT. Accordingly, we model the offloading scheduling process as a Markov Decision Process (MDP) to minimize the long-term cost in terms of a trade off between task latency, energy consumption, and renting cost of clouds. Simulation results demonstrate that our algorithm can effectively find the optimal offloading strategy, as well as achieve the fast convergence speed and high performance, compared with other existing approaches.

See the Near Future: A Short-Term Predictive Methodology to Traffic Load in ITS

The Intelligent Transportation System (ITS) targets to a coordinated traffic system by applying the advanced wireless communication technologies for road traffic scheduling. Towards an accurate road traffic control, the short-term traffic forecasting to predict the road traffic at the particular site in a short period is often useful and important. In existing works, Seasonal Autoregressive Integrated Moving Average (SARIMA) model is a popular approach. The scheme however encounters two challenges: 1) the analysis on related data is insufficient whereas some important features of data may be neglected; and 2) with data presenting different features, it is unlikely to have one predictive model that can fit all situations. To tackle above issues, in this work, we develop a hybrid model to improve accuracy of SARIMA. In specific, we first explore the autocorrelation and distribution features existed in traffic flow to revise structure of the time series model. Based on the Gaussian distribution of traffic flow, a hybrid model with a Bayesian learning algorithm is developed which can effectively expand the application scenarios of SARIMA. We show the efficiency and accuracy of our proposal using both analysis and experimental studies. Using the real-world trace data, we show that the proposed predicting approach can achieve satisfactory performance in practice.