Learning-Based UAV Path Planning for Data Collection with Integrated Collision Avoidance

Unmanned aerial vehicles (UAVs) are expected to be an integral part of wireless networks, and determining collision-free trajectory in multi-UAV non-cooperative scenarios while collecting data from distributed Internet of Things (IoT) nodes is a challenging task. In this paper, we consider a path planning optimization problem to maximize the collected data from multiple IoT nodes under realistic constraints. The considered multi-UAV non-cooperative scenarios involve random number of other UAVs in addition to the typical UAV, and UAVs do not communicate or share information among each other. We translate the problem into a Markov decision process (MDP) with parameterized states, permissible actions, and detailed reward functions. Dueling double deep Q-network (D3QN) is proposed to learn the decision making policy for the typical UAV, without any prior knowledge of the environment (e.g., channel propagation model and locations of the obstacles) and other UAVs (e.g., their missions, movements, and policies). The proposed algorithm can adapt to various missions in various scenarios, e.g., different numbers and positions of IoT nodes, different amount of data to be collected, and different numbers and positions of other UAVs. Numerical results demonstrate that real-time navigation can be efficiently performed with high success rate, high data collection rate, and low collision rate.

Learning-Based UAV Trajectory Optimization with Collision Avoidance and Connectivity Constraints

Unmanned aerial vehicles (UAVs) are expected to be an integral part of wireless networks, and determining collision-free trajectories for multiple UAVs while satisfying requirements of connectivity with ground base stations (GBSs) is a challenging task. In this paper, we first reformulate the multi-UAV trajectory optimization problem with collision avoidance and wireless connectivity constraints as a sequential decision making problem in the discrete time domain. We, then, propose a decentralized deep reinforcement learning approach to solve the problem. More specifically, a value network is developed to encode the expected time to destination given the agent's joint state (including the agent's information, the nearby agents' observable information, and the locations of the nearby GBSs). A signal-to-interference-plus-noise ratio (SINR)-prediction neural network is also designed, using accumulated SINR measurements obtained when interacting with the cellular network, to map the GBSs' locations into the SINR levels in order to predict the UAV's SINR. Numerical results show that with the value network and SINR-prediction network, real-time navigation for multi-UAVs can be efficiently performed in various environments with high success rate.

Uplink Coverage in Heterogeneous mmWave Cellular Networks with Clustered Users

A K-tier heterogeneous mmWave uplink cellular network with clustered user equipments (UEs) is considered in this paper. In particular, UEs are assumed to be clustered around small-cell base stations (BSs) according to a Gaussian distribution, leading to the Thomas cluster process based modeling. Specific and practical line-of-sight (LOS) and non-line-of-sight (NLOS) models are adopted with different parameters for different tiers. The probability density functions (PDFs) and complementary cumulative distribution functions (CCDFs) of different distances from UEs to BSs are characterized. Coupled association strategy and largest long-term averaged biased received power criterion are considered, and general expressions for association probabilities are provided. Following the identification of the association probabilities, the Laplace transforms of the inter-cell interference and the intra-cluster interference are characterized. Using tools from stochastic geometry, general expressions of the SINR coverage probability are provided. As extensions, fractional power control is incorporated into the analysis, tractable closed-form expressions are provided for special cases, and average ergodic spectral efficiency is analyzed. Via numerical and simulation results, analytical characterizations are confirmed and the impact of key system and network parameters on the performance is identified.

Jamming-Resilient Path Planning for Multiple UAVs via Deep Reinforcement Learning

Unmanned aerial vehicles (UAVs) are expected to be an integral part of wireless networks. In this paper, we aim to find collision-free paths for multiple cellular-connected UAVs, while satisfying requirements of connectivity with ground base stations (GBSs) in the presence of a dynamic jammer. We first formulate the problem as a sequential decision making problem in discrete domain, with connectivity, collision avoidance, and kinematic constraints. We, then, propose an offline temporal difference (TD) learning algorithm with online signal-to-interference-plus-noise ratio (SINR) mapping to solve the problem. More specifically, a value network is constructed and trained offline by TD method to encode the interactions among the UAVs and between the UAVs and the environment; and an online SINR mapping deep neural network (DNN) is designed and trained by supervised learning, to encode the influence and changes due to the jammer. Numerical results show that, without any information on the jammer, the proposed algorithm can achieve performance levels close to that of the ideal scenario with the perfect SINR-map. Real-time navigation for multi-UAVs can be efficiently performed with high success rates, and collisions are avoided.