A Decision-Making GPT Model Augmented with Entropy Regularization for Autonomous Vehicles

In the domain of autonomous vehicles (AVs), decision-making is a critical factor that significantly influences the efficacy of autonomous navigation. As the field progresses, the enhancement of decision-making capabilities in complex environments has become a central area of research within data-driven methodologies. Despite notable advances, existing learning-based decision-making strategies in autonomous vehicles continue to reveal opportunities for further refinement, particularly in the articulation of policies and the assurance of safety. In this study, the decision-making challenges associated with autonomous vehicles are conceptualized through the framework of the Constrained Markov Decision Process (CMDP) and approached as a sequence modeling problem. Utilizing the Generative Pre-trained Transformer (GPT), we introduce a novel decision-making model tailored for AVs, which incorporates entropy regularization techniques to bolster exploration and enhance safety performance. Comprehensive experiments conducted across various scenarios affirm that our approach surpasses several established baseline methods, particularly in terms of safety and overall efficacy.

Generalizable Implicit Neural Representation As a Universal Spatiotemporal Traffic Data Learner

$\textbf{This is the conference version of our paper: Spatiotemporal Implicit Neural Representation as a Generalized Traffic Data Learner}$. Spatiotemporal Traffic Data (STTD) measures the complex dynamical behaviors of the multiscale transportation system. Existing methods aim to reconstruct STTD using low-dimensional models. However, they are limited to data-specific dimensions or source-dependent patterns, restricting them from unifying representations. Here, we present a novel paradigm to address the STTD learning problem by parameterizing STTD as an implicit neural representation. To discern the underlying dynamics in low-dimensional regimes, coordinate-based neural networks that can encode high-frequency structures are employed to directly map coordinates to traffic variables. To unravel the entangled spatial-temporal interactions, the variability is decomposed into separate processes. We further enable modeling in irregular spaces such as sensor graphs using spectral embedding. Through continuous representations, our approach enables the modeling of a variety of STTD with a unified input, thereby serving as a generalized learner of the underlying traffic dynamics. It is also shown that it can learn implicit low-rank priors and smoothness regularization from the data, making it versatile for learning different dominating data patterns. We validate its effectiveness through extensive experiments in real-world scenarios, showcasing applications from corridor to network scales. Empirical results not only indicate that our model has significant superiority over conventional low-rank models, but also highlight that the versatility of the approach. We anticipate that this pioneering modeling perspective could lay the foundation for universal representation of STTD in various real-world tasks. $\textbf{The full version can be found at:}$ https://doi.org/10.48550/arXiv.2405.03185.

Towards Interactive Autonomous Vehicle Testing: Vehicle-Under-Test-Centered Traffic Simulation

The simulation-based testing is essential for safely implementing autonomous vehicles (AVs) on roads, necessitating simulated traffic environments that dynamically interact with the Vehicle Under Test (VUT). This study introduces a VUT-Centered environmental Dynamics Inference (VCDI) model for realistic, interactive, and diverse background traffic simulation. VCDI is built on a Transformer-based trajectory inference model to generate trajectories for background objects. Serving the purpose of AV testing, VCDI additionally considers VUT-centered interactivity and scenario diversity using a conditional inference framework. First, the VUT future motion is taken as an augmented model input to bridge the interaction between VUT and background objects. Second, to enrich the scenario diversity, a Bayesian-network-based cost function module is designed. The module, learned in a distributional form, captures the uncertainty of the VUT's strategy, triggering various scenario evolution. Experimental results validate VCDI's trajectory-level simulation precision which outperforms the state-of-the-art trajectory prediction work. The flexibility of the distributional cost function allows VCDI to provide diverse-yet-realistic scenarios for AV testing. We demonstrate such capability by modifying the anticipation to VUT's cost-based strategy and thus achieve multiple testing scenarios with explainable background traffic evolution.

Evaluation of Drivers' Interaction Ability at Social Scenarios: A Process-Based Framework

Assessing drivers' interaction capabilities is crucial for understanding human driving behavior and enhancing the interactive abilities of autonomous vehicles. In scenarios involving strong interaction, existing metrics focused on interaction outcomes struggle to capture the evolutionary process of drivers' interactive behaviors, making it challenging for autonomous vehicles to dynamically assess and respond to other agents during interactions. To address this issue, we propose a framework for assessing drivers' interaction capabilities, oriented towards the interactive process itself, which includes three components: Interaction Risk Perception, Interaction Process Modeling, and Interaction Ability Scoring. We quantify interaction risks through motion state estimation and risk field theory, followed by introducing a dynamic action assessment benchmark based on a game-theoretical rational agent model, and designing a capability scoring metric based on morphological similarity distance. By calculating real-time differences between a driver's actions and the assessment benchmark, the driver's interaction capabilities are scored dynamically. We validated our framework at unsignalized intersections as a typical scenario. Validation analysis on driver behavior datasets from China and the USA shows that our framework effectively distinguishes and evaluates conservative and aggressive driving states during interactions, demonstrating good adaptability and effectiveness in various regional settings.

Residual-Conditioned Optimal Transport: Towards Structure-preserving Unpaired and Paired Image Restoration

Deep learning-based image restoration methods have achieved promising performance. However, how to faithfully preserve the structure of the original image remains challenging. To address this challenge, we propose a novel Residual-Conditioned Optimal Transport (RCOT) approach, which models the image restoration as an optimal transport (OT) problem for both unpaired and paired settings, integrating the transport residual as a unique degradation-specific cue for both the transport cost and the transport map. Specifically, we first formalize a Fourier residual-guided OT objective by incorporating the degradation-specific information of the residual into the transport cost. Based on the dual form of the OT formulation, we design the transport map as a two-pass RCOT map that comprises a base model and a refinement process, in which the transport residual is computed by the base model in the first pass and then encoded as a degradation-specific embedding to condition the second-pass restoration. By duality, the RCOT problem is transformed into a minimax optimization problem, which can be solved by adversarially training neural networks. Extensive experiments on multiple restoration tasks show the effectiveness of our approach in terms of both distortion measures and perceptual quality. Particularly, RCOT restores images with more faithful structural details compared to state-of-the-art methods.

Adversarial Reweighting with $α$-Power Maximization for Domain Adaptation

The practical Domain Adaptation (DA) tasks, e.g., Partial DA (PDA), open-set DA, universal DA, and test-time adaptation, have gained increasing attention in the machine learning community. In this paper, we propose a novel approach, dubbed Adversarial Reweighting with $\alpha$-Power Maximization (ARPM), for PDA where the source domain contains private classes absent in target domain. In ARPM, we propose a novel adversarial reweighting model that adversarially learns to reweight source domain data to identify source-private class samples by assigning smaller weights to them, for mitigating potential negative transfer. Based on the adversarial reweighting, we train the transferable recognition model on the reweighted source distribution to be able to classify common class data. To reduce the prediction uncertainty of the recognition model on the target domain for PDA, we present an $\alpha$-power maximization mechanism in ARPM, which enriches the family of losses for reducing the prediction uncertainty for PDA. Extensive experimental results on five PDA benchmarks, i.e., Office-31, Office-Home, VisDA-2017, ImageNet-Caltech, and DomainNet, show that our method is superior to recent PDA methods. Ablation studies also confirm the effectiveness of components in our approach. To theoretically analyze our method, we deduce an upper bound of target domain expected error for PDA, which is approximately minimized in our approach. We further extend ARPM to open-set DA, universal DA, and test time adaptation, and verify the usefulness through experiments.

Data Quality Matters: Suicide Intention Detection on Social Media Posts Using a RoBERTa-CNN Model

Suicide remains a global health concern for the field of health, which urgently needs innovative approaches for early detection and intervention. In this paper, we focus on identifying suicidal intentions in SuicideWatch Reddit posts and present a novel approach to suicide detection using the cutting-edge RoBERTa-CNN model, a variant of RoBERTa (Robustly optimized BERT approach). RoBERTa is used for various Natural Language Processing (NLP) tasks, including text classification and sentiment analysis. The effectiveness of the RoBERTa lies in its ability to capture textual information and form semantic relationships within texts. By adding the Convolution Neural Network (CNN) layer to the original model, the RoBERTa enhances its ability to capture important patterns from heavy datasets. To evaluate the RoBERTa-CNN, we experimented on the Suicide and Depression Detection dataset and obtained solid results. For example, RoBERTa-CNN achieves 98% mean accuracy with the standard deviation (STD) of 0.0009. It also reaches over 97.5% mean AUC value with an STD of 0.0013. In the meanwhile, RoBERTa-CNN outperforms competitive methods, demonstrating the robustness and ability to capture nuanced linguistic patterns for suicidal intentions. Therefore, RoBERTa-CNN can detect suicide intention on text data very well.

Floorplanning of VLSI by Mixed-Variable Optimization

By formulating the floorplanning of VLSI as a mixed-variable optimization problem, this paper proposes to solve it by memetic algorithms, where the discrete orientation variables are addressed by the distribution evolutionary algorithm based on a population of probability model (DEA-PPM), and the continuous coordination variables are optimized by the conjugate sub-gradient algorithm (CSA). Accordingly, the fixed-outline floorplanning algorithm based on CSA and DEA-PPM (FFA-CD) and the floorplanning algorithm with golden section strategy (FA-GSS) are proposed for the floorplanning problems with and without fixed-outline constraint. %FF-CD is committed to optimizing wirelength targets within a fixed profile. FA-GSS uses the Golden Section strategy to optimize both wirelength and area targets. The CSA is used to solve the proposed non-smooth optimization model, and the DEA-PPM is used to explore the module rotation scheme to enhance the flexibility of the algorithm. Numerical experiments on GSRC test circuits show that the proposed algorithms are superior to some celebrated B*-tree based floorplanning algorithms, and are expected to be applied to large-scale floorplanning problems due to their low time complexity.

Learning Hybrid Policies for MPC with Application to Drone Flight in Unknown Dynamic Environments

In recent years, drones have found increased applications in a wide array of real-world tasks. Model predictive control (MPC) has emerged as a practical method for drone flight control, owing to its robustness against modeling errors/uncertainties and external disturbances. However, MPC's sensitivity to manually tuned parameters can lead to rapid performance degradation when faced with unknown environmental dynamics. This paper addresses the challenge of controlling a drone as it traverses a swinging gate characterized by unknown dynamics. This paper introduces a parameterized MPC approach named hyMPC that leverages high-level decision variables to adapt to uncertain environmental conditions. To derive these decision variables, a novel policy search framework aimed at training a high-level Gaussian policy is presented. Subsequently, we harness the power of neural network policies, trained on data gathered through the repeated execution of the Gaussian policy, to provide real-time decision variables. The effectiveness of hyMPC is validated through numerical simulations, achieving a 100\% success rate in 20 drone flight tests traversing a swinging gate, demonstrating its capability to achieve safe and precise flight with limited prior knowledge of environmental dynamics.

DDM-Lag : A Diffusion-based Decision-making Model for Autonomous Vehicles with Lagrangian Safety Enhancement

Decision-making stands as a pivotal component in the realm of autonomous vehicles (AVs), playing a crucial role in navigating the intricacies of autonomous driving. Amidst the evolving landscape of data-driven methodologies, enhancing decision-making performance in complex scenarios has emerged as a prominent research focus. Despite considerable advancements, current learning-based decision-making approaches exhibit potential for refinement, particularly in aspects of policy articulation and safety assurance. To address these challenges, we introduce DDM-Lag, a Diffusion Decision Model,augmented with Lagrangian-based safety enhancements.In our approach, the autonomous driving decision-making conundrum is conceptualized as a Constrained Markov Decision Process (CMDP). We have crafted an Actor-Critic framework, wherein the diffusion model is employed as the actor,facilitating policy exploration and learning. The integration of safety constraints in the CMDP and the adoption of a Lagrangian relaxation-based policy optimization technique ensure enhanced decision safety. A PID controller is employed for the stable updating of model parameters. The effectiveness of DDM-Lag is evaluated through different driving tasks, showcasing improvements in decision-making safety and overall performance compared to baselines.