Attention-based Learning for 3D Informative Path Planning

In this work, we propose an attention-based deep reinforcement learning approach to address the adaptive informative path planning (IPP) problem in 3D space, where an aerial robot equipped with a downward-facing sensor must dynamically adjust its 3D position to balance sensing footprint and accuracy, and finally obtain a high-quality belief of an underlying field of interest over a given domain (e.g., presence of specific plants, hazardous gas, geological structures, etc.). In adaptive IPP tasks, the agent is tasked with maximizing information collected under time/distance constraints, continuously adapting its path based on newly acquired sensor data. To this end, we leverage attention mechanisms for their strong ability to capture global spatial dependencies across large action spaces, allowing the agent to learn an implicit estimation of environmental transitions. Our model builds a contextual belief representation over the entire domain, guiding sequential movement decisions that optimize both short- and long-term search objectives. Comparative evaluations against state-of-the-art planners demonstrate that our approach significantly reduces environmental uncertainty within constrained budgets, thus allowing the agent to effectively balance exploration and exploitation. We further show our model generalizes well to environments of varying sizes, highlighting its potential for many real-world applications.

Hybrid Beamforming Optimization for MIMO ISAC Exploiting Prior Information: A PCRB-based Approach

This paper considers a multiple-input multiple-output (MIMO) integrated sensing and communication (ISAC) system, where a multi-antenna base station (BS) with transceiver hybrid analog-digital arrays transmits dual-functional signals to communicate with a multi-antenna user and simultaneously sense the unknown and random location information of a target based on the reflected echo signals and the prior distribution information on the target's location. Under transceiver hybrid arrays, we characterize the sensing performance by deriving the posterior Cram\'{e}r-Rao bound (PCRB) of the mean-squared error which is a function of the transmit hybrid beamforming and receive analog beamforming. We study joint transmit hybrid beamforming and receive analog beamforming optimization to minimize the PCRB subject to a communication rate requirement. We first consider a sensing-only system and derive the optimal solution to each element in the transmit/receive analog beamforming matrices that minimizes the PCRB in closed form. Then, we develop an alternating optimization (AO) based algorithm. Next, we study a narrowband MIMO ISAC system and devise an efficient AO-based hybrid beamforming algorithm by leveraging weighted minimum mean-squared error and feasible point pursuit successive convex approximation methods. Furthermore, we extend the results for narrowband systems to a MIMO orthogonal frequency-division multiplexing (OFDM) ISAC system. Numerical results validate the effectiveness of our proposed hybrid beamforming designs. It is revealed that the number of receive RF chains has more significant impact on the sensing performance than its transmit counterpart. Under a given budget on the total number of transmit/receive RF chains at the BS, the optimal number of transmit RF chains increases as the communication rate target increases due to the non-trivial PCRB-rate trade-off.

Hybrid Beamforming Design for Integrated Sensing and Communication Exploiting Prior Information

In this paper, we investigate the hybrid beamforming design for a multiple-input multiple-output (MIMO) integrated sensing and communication (ISAC) system, where a multi-antenna base station (BS) with hybrid analog-digital transmit antenna arrays sends dual-functional signals to communicate with a multi-antenna user and simultaneously sense the location information of a point target based on the reflected echo signals. Specifically, we aim to sense the target's unknown and random angle information by exploiting its prior distribution information, with posterior Cram\'{e}r-Rao bound (PCRB) employed as the sensing performance metric. First, we consider a sensing-only case and study the hybrid beamforming optimization to minimize the sensing PCRB. We analytically prove that hybrid beamforming can achieve the same performance as the optimized digital beamforming as long as the number of radio frequency (RF) chains is larger than 1. Then, we propose a convex relaxation based algorithm for the hybrid beamforming design with a single RF chain. Next, we study the hybrid beamforming optimization to minimize the PCRB subject to a communication rate target. Due to the intractability of the exact PCRB expression, we replace it with a tight upper bound. Although this problem is still non-convex and challenging to solve, we propose an alternating optimization (AO) algorithm for finding a high-quality suboptimal solution based on the feasible point pursuit successive convex approximation (FPP-SCA) method. Numerical results validate the effectiveness of our proposed hybrid beamforming design.

Deep Reinforcement Learning-based Large-scale Robot Exploration

In this work, we propose a deep reinforcement learning (DRL) based reactive planner to solve large-scale Lidar-based autonomous robot exploration problems in 2D action space. Our DRL-based planner allows the agent to reactively plan its exploration path by making implicit predictions about unknown areas, based on a learned estimation of the underlying transition model of the environment. To this end, our approach relies on learned attention mechanisms for their powerful ability to capture long-term dependencies at different spatial scales to reason about the robot's entire belief over known areas. Our approach relies on ground truth information (i.e., privileged learning) to guide the environment estimation during training, as well as on a graph rarefaction algorithm, which allows models trained in small-scale environments to scale to large-scale ones. Simulation results show that our model exhibits better exploration efficiency (12% in path length, 6% in makespan) and lower planning time (60%) than the state-of-the-art planners in a 130m x 100m benchmark scenario. We also validate our learned model on hardware.

Spatio-Temporal Attention Network for Persistent Monitoring of Multiple Mobile Targets

This work focuses on the persistent monitoring problem, where a set of targets moving based on an unknown model must be monitored by an autonomous mobile robot with a limited sensing range. To keep each target's position estimate as accurate as possible, the robot needs to adaptively plan its path to (re-)visit all the targets and update its belief from measurements collected along the way. In doing so, the main challenge is to strike a balance between exploitation, i.e., re-visiting previously-located targets, and exploration, i.e., finding new targets or re-acquiring lost ones. Encouraged by recent advances in deep reinforcement learning, we introduce an attention-based neural solution to the persistent monitoring problem, where the agent can learn the inter-dependencies between targets, i.e., their spatial and temporal correlations, conditioned on past measurements. This endows the agent with the ability to determine which target, time, and location to attend to across multiple scales, which we show also helps relax the usual limitations of a finite target set. We experimentally demonstrate that our method outperforms other baselines in terms of number of targets visits and average estimation error in complex environments. Finally, we implement and validate our model in a drone-based simulation experiment to monitor mobile ground targets in a high-fidelity simulator.

ARiADNE: A Reinforcement learning approach using Attention-based Deep Networks for Exploration

In autonomous robot exploration tasks, a mobile robot needs to actively explore and map an unknown environment as fast as possible. Since the environment is being revealed during exploration, the robot needs to frequently re-plan its path online, as new information is acquired by onboard sensors and used to update its partial map. While state-of-the-art exploration planners are frontier- and sampling-based, encouraged by the recent development in deep reinforcement learning (DRL), we propose ARiADNE, an attention-based neural approach to obtain real-time, non-myopic path planning for autonomous exploration. ARiADNE is able to learn dependencies at multiple spatial scales between areas of the agent's partial map, and implicitly predict potential gains associated with exploring those areas. This allows the agent to sequence movement actions that balance the natural trade-off between exploitation/refinement of the map in known areas and exploration of new areas. We experimentally demonstrate that our method outperforms both learning and non-learning state-of-the-art baselines in terms of average trajectory length to complete exploration in hundreds of simplified 2D indoor scenarios. We further validate our approach in high-fidelity Robot Operating System (ROS) simulations, where we consider a real sensor model and a realistic low-level motion controller, toward deployment on real robots.

DL-Traff: Survey and Benchmark of Deep Learning Models for Urban Traffic Prediction

Nowadays, with the rapid development of IoT (Internet of Things) and CPS (Cyber-Physical Systems) technologies, big spatiotemporal data are being generated from mobile phones, car navigation systems, and traffic sensors. By leveraging state-of-the-art deep learning technologies on such data, urban traffic prediction has drawn a lot of attention in AI and Intelligent Transportation System community. The problem can be uniformly modeled with a 3D tensor (T, N, C), where T denotes the total time steps, N denotes the size of the spatial domain (i.e., mesh-grids or graph-nodes), and C denotes the channels of information. According to the specific modeling strategy, the state-of-the-art deep learning models can be divided into three categories: grid-based, graph-based, and multivariate time-series models. In this study, we first synthetically review the deep traffic models as well as the widely used datasets, then build a standard benchmark to comprehensively evaluate their performances with the same settings and metrics. Our study named DL-Traff is implemented with two most popular deep learning frameworks, i.e., TensorFlow and PyTorch, which is already publicly available as two GitHub repositories https://github.com/deepkashiwa20/DL-Traff-Grid and https://github.com/deepkashiwa20/DL-Traff-Graph. With DL-Traff, we hope to deliver a useful resource to researchers who are interested in spatiotemporal data analysis.