Interpretable Long Term Waypoint-Based Trajectory Prediction Model

Predicting the future trajectories of dynamic agents in complex environments is crucial for a variety of applications, including autonomous driving, robotics, and human-computer interaction. It is a challenging task as the behavior of the agent is unknown and intrinsically multimodal. Our key insight is that the agents behaviors are influenced not only by their past trajectories and their interaction with their immediate environment but also largely with their long term waypoint (LTW). In this paper, we study the impact of adding a long-term goal on the performance of a trajectory prediction framework. We present an interpretable long term waypoint-driven prediction framework (WayDCM). WayDCM first predict an agent's intermediate goal (IG) by encoding his interactions with the environment as well as his LTW using a combination of a Discrete choice Model (DCM) and a Neural Network model (NN). Then, our model predicts the corresponding trajectories. This is in contrast to previous work which does not consider the ultimate intent of the agent to predict his trajectory. We evaluate and show the effectiveness of our approach on the Waymo Open dataset.

Hierarchical Attention and Graph Neural Networks: Toward Drift-Free Pose Estimation

The most commonly used method for addressing 3D geometric registration is the iterative closet-point algorithm, this approach is incremental and prone to drift over multiple consecutive frames. The Common strategy to address the drift is the pose graph optimization subsequent to frame-to-frame registration, incorporating a loop closure process that identifies previously visited places. In this paper, we explore a framework that replaces traditional geometric registration and pose graph optimization with a learned model utilizing hierarchical attention mechanisms and graph neural networks. We propose a strategy to condense the data flow, preserving essential information required for the precise estimation of rigid poses. Our results, derived from tests on the KITTI Odometry dataset, demonstrate a significant improvement in pose estimation accuracy. This improvement is especially notable in determining rotational components when compared with results obtained through conventional multi-way registration via pose graph optimization. The code will be made available upon completion of the review process.

Interpretable Goal-Based model for Vehicle Trajectory Prediction in Interactive Scenarios

The abilities to understand the social interaction behaviors between a vehicle and its surroundings while predicting its trajectory in an urban environment are critical for road safety in autonomous driving. Social interactions are hard to explain because of their uncertainty. In recent years, neural network-based methods have been widely used for trajectory prediction and have been shown to outperform hand-crafted methods. However, these methods suffer from their lack of interpretability. In order to overcome this limitation, we combine the interpretability of a discrete choice model with the high accuracy of a neural network-based model for the task of vehicle trajectory prediction in an interactive environment. We implement and evaluate our model using the INTERACTION dataset and demonstrate the effectiveness of our proposed architecture to explain its predictions without compromising the accuracy.

Trajectory Prediction for Autonomous Driving based on Multi-Head Attention with Joint Agent-Map Representation

Predicting the trajectories of surrounding agents is an essential ability for robots navigating complex real-world environments. Autonomous vehicles (AV) in particular, can generate safe and efficient path plans by predicting the motion of surrounding road users. Future trajectories of agents can be inferred using two tightly linked cues: the locations and past motion of agents, and the static scene structure. The configuration of the agents may uncover which part of the scene is more relevant, while the scene structure can determine the relative influence of agents on each other's motion. To better model the interdependence of the two cues, we propose a multi-head attention-based model that uses a joint representation of the static scene and agent configuration for generating both keys and values for the attention heads. Moreover, to address the multimodality of future agent motion, we propose to use each attention head to generate a distinct future trajectory of the agent. Our model achieves state of the art results on the publicly available nuScenes dataset and generates diverse future trajectories compliant with scene structure and agent configuration. Additionally, the visualization of attention maps adds a layer of interpretability to the trajectories predicted by the model.

Multi-Head Attention with Joint Agent-Map Representation for Trajectory Prediction in Autonomous Driving

For autonomous vehicles to navigate in urban environment, the ability to predict the possible future behaviors of surrounding vehicles is essential to increase their safety level by avoiding dangerous situations in advance. The behavior anticipation task is mainly based on two tightly linked cues; surrounding agents' recent motions and scene information. The configuration of the agents may uncover which part of the scene is important, while scene structure determines the influential existing agents. To better present this correlation, we deploy multi-head attention on a joint agents and map context. Moreover, to account for the uncertainty of the future, we use an efficient multi-modal probabilistic trajectory prediction model that learns to extract different joint context features and generate diverse possible trajectories accordingly in one forward pass. Results on the publicly available nuScenes dataset prove that our model achieves the performance of existing methods and generates diverse possible future trajectories compliant with scene structure. Most importantly, the visualization of attention maps reveals some of the underlying prediction logic of our approach which increases its interpretability and reliability to deploy in the real world.

End-to-End Race Driving with Deep Reinforcement Learning

We present research using the latest reinforcement learning algorithm for end-to-end driving without any mediated perception (object recognition, scene understanding). The newly proposed reward and learning strategies lead together to faster convergence and more robust driving using only RGB image from a forward facing camera. An Asynchronous Actor Critic (A3C) framework is used to learn the car control in a physically and graphically realistic rally game, with the agents evolving simultaneously on tracks with a variety of road structures (turns, hills), graphics (seasons, location) and physics (road adherence). A thorough evaluation is conducted and generalization is proven on unseen tracks and using legal speed limits. Open loop tests on real sequences of images show some domain adaption capability of our method.

Sparse and Dense Data with CNNs: Depth Completion and Semantic Segmentation

Convolutional neural networks are designed for dense data, but vision data is often sparse (stereo depth, point clouds, pen stroke, etc.). We present a method to handle sparse depth data with optional dense RGB, and accomplish depth completion and semantic segmentation changing only the last layer. Our proposal efficiently learns sparse features without the need of an additional validity mask. We show how to ensure network robustness to varying input sparsities. Our method even works with densities as low as 0.8% (8 layer lidar), and outperforms all published state-of-the-art on the Kitti depth completion benchmark.

Low Speed Automation, a French Initiative

Nowadays, vehicle safety is constantly increasing thanks to the improvement of vehicle passive and active safety. However, on a daily usage of the car, traffic jams remains a problem. With limited space for road infrastructure, automation of the driving task on specific situation seems to be a possible solution. The French project ABV, which stands for low speed automation, tries to demonstrate the feasibility of the concept and to prove the benefits. In this article, we describe the scientific background of the project and expected outputs.

An ontology-based approach to relax traffic regulation for autonomous vehicle assistance

Traffic regulation must be respected by all vehicles, either human- or computer- driven. However, extreme traffic situations might exhibit practical cases in which a vehicle should safely and reasonably relax traffic regulation, e.g., in order not to be indefinitely blocked and to keep circulating. In this paper, we propose a high-level representation of an automated vehicle, other vehicles and their environment, which can assist drivers in taking such "illegal" but practical relaxation decisions. This high-level representation (an ontology) includes topological knowledge and inference rules, in order to compute the next high-level motion an automated vehicle should take, as assistance to a driver. Results on practical cases are presented.

Joint interpretation of on-board vision and static GPS cartography for determination of correct speed limit

We present here a first prototype of a "Speed Limit Support" Advance Driving Assistance System (ADAS) producing permanent reliable information on the current speed limit applicable to the vehicle. Such a module can be used either for information of the driver, or could even serve for automatic setting of the maximum speed of a smart Adaptive Cruise Control (ACC). Our system is based on a joint interpretation of cartographic information (for static reference information) with on-board vision, used for traffic sign detection and recognition (including supplementary sub-signs) and visual road lines localization (for detection of lane changes). The visual traffic sign detection part is quite robust (90% global correct detection and recognition for main speed signs, and 80% for exit-lane sub-signs detection). Our approach for joint interpretation with cartography is original, and logic-based rather than probability-based, which allows correct behaviour even in cases, which do happen, when both vision and cartography may provide the same erroneous information.