Alert button
Picture for Hua Wei

Hua Wei

Alert button

Towards Robust Fidelity for Evaluating Explainability of Graph Neural Networks

Oct 03, 2023
Xu Zheng, Farhad Shirani, Tianchun Wang, Wei Cheng, Zhuomin Chen, Haifeng Chen, Hua Wei, Dongsheng Luo

Figure 1 for Towards Robust Fidelity for Evaluating Explainability of Graph Neural Networks
Figure 2 for Towards Robust Fidelity for Evaluating Explainability of Graph Neural Networks
Figure 3 for Towards Robust Fidelity for Evaluating Explainability of Graph Neural Networks
Figure 4 for Towards Robust Fidelity for Evaluating Explainability of Graph Neural Networks

Graph Neural Networks (GNNs) are neural models that leverage the dependency structure in graphical data via message passing among the graph nodes. GNNs have emerged as pivotal architectures in analyzing graph-structured data, and their expansive application in sensitive domains requires a comprehensive understanding of their decision-making processes -- necessitating a framework for GNN explainability. An explanation function for GNNs takes a pre-trained GNN along with a graph as input, to produce a `sufficient statistic' subgraph with respect to the graph label. A main challenge in studying GNN explainability is to provide fidelity measures that evaluate the performance of these explanation functions. This paper studies this foundational challenge, spotlighting the inherent limitations of prevailing fidelity metrics, including $Fid_+$, $Fid_-$, and $Fid_\Delta$. Specifically, a formal, information-theoretic definition of explainability is introduced and it is shown that existing metrics often fail to align with this definition across various statistical scenarios. The reason is due to potential distribution shifts when subgraphs are removed in computing these fidelity measures. Subsequently, a robust class of fidelity measures are introduced, and it is shown analytically that they are resilient to distribution shift issues and are applicable in a wide range of scenarios. Extensive empirical analysis on both synthetic and real datasets are provided to illustrate that the proposed metrics are more coherent with gold standard metrics.

* 23 Pages, 10 figures, under review 
Viaarxiv icon

Uncertainty-aware Traffic Prediction under Missing Data

Sep 15, 2023
Hao Mei, Junxian Li, Zhiming Liang, Guanjie Zheng, Bin Shi, Hua Wei

Figure 1 for Uncertainty-aware Traffic Prediction under Missing Data
Figure 2 for Uncertainty-aware Traffic Prediction under Missing Data
Figure 3 for Uncertainty-aware Traffic Prediction under Missing Data
Figure 4 for Uncertainty-aware Traffic Prediction under Missing Data

Traffic prediction is a crucial topic because of its broad scope of applications in the transportation domain. Recently, various studies have achieved promising results. However, most studies assume the prediction locations have complete or at least partial historical records and cannot be extended to non-historical recorded locations. In real-life scenarios, the deployment of sensors could be limited due to budget limitations and installation availability, which makes most current models not applicable. Though few pieces of literature tried to impute traffic states at the missing locations, these methods need the data simultaneously observed at the locations with sensors, making them not applicable to prediction tasks. Another drawback is the lack of measurement of uncertainty in prediction, making prior works unsuitable for risk-sensitive tasks or involving decision-making. To fill the gap, inspired by the previous inductive graph neural network, this work proposed an uncertainty-aware framework with the ability to 1) extend prediction to missing locations with no historical records and significantly extend spatial coverage of prediction locations while reducing deployment of sensors and 2) generate probabilistic prediction with uncertainty quantification to help the management of risk and decision making in the down-stream tasks. Through extensive experiments on real-life datasets, the result shows our method achieved promising results on prediction tasks, and the uncertainty quantification gives consistent results which highly correlated with the locations with and without historical data. We also show that our model could help support sensor deployment tasks in the transportation field to achieve higher accuracy with a limited sensor deployment budget.

* 11 pages, 3 figures, Accepted as a short paper of IEEE International Conference on Data Mining (ICDM) 2023 
Viaarxiv icon

LLM Powered Sim-to-real Transfer for Traffic Signal Control

Sep 04, 2023
Longchao Da, Minchiuan Gao, Hao Mei, Hua Wei

Figure 1 for LLM Powered Sim-to-real Transfer for Traffic Signal Control
Figure 2 for LLM Powered Sim-to-real Transfer for Traffic Signal Control
Figure 3 for LLM Powered Sim-to-real Transfer for Traffic Signal Control
Figure 4 for LLM Powered Sim-to-real Transfer for Traffic Signal Control

Numerous solutions are proposed for the Traffic Signal Control (TSC) tasks aiming to provide efficient transportation and mitigate congestion waste. In recent, promising results have been attained by Reinforcement Learning (RL) methods through trial and error in simulators, bringing confidence in solving cities' congestion headaches. However, there still exist performance gaps when simulator-trained policies are deployed to the real world. This issue is mainly introduced by the system dynamic difference between the training simulator and the real-world environments. The Large Language Models (LLMs) are trained on mass knowledge and proved to be equipped with astonishing inference abilities. In this work, we leverage LLMs to understand and profile the system dynamics by a prompt-based grounded action transformation. Accepting the cloze prompt template, and then filling in the answer based on accessible context, the pre-trained LLM's inference ability is exploited and applied to understand how weather conditions, traffic states, and road types influence traffic dynamics, being aware of this, the policies' action is taken and grounded based on realistic dynamics, thus help the agent learn a more realistic policy. We conduct experiments using DQN to show the effectiveness of the proposed PromptGAT's ability in mitigating the performance gap from simulation to reality (sim-to-real).

* 9 pages, 7 figures 
Viaarxiv icon

Safety in Traffic Management Systems: A Comprehensive Survey

Aug 11, 2023
Wenlu Du, Ankan Dash, Jing Li, Hua Wei, Guiling Wang

Figure 1 for Safety in Traffic Management Systems: A Comprehensive Survey
Figure 2 for Safety in Traffic Management Systems: A Comprehensive Survey
Figure 3 for Safety in Traffic Management Systems: A Comprehensive Survey
Figure 4 for Safety in Traffic Management Systems: A Comprehensive Survey

Traffic management systems play a vital role in ensuring safe and efficient transportation on roads. However, the use of advanced technologies in traffic management systems has introduced new safety challenges. Therefore, it is important to ensure the safety of these systems to prevent accidents and minimize their impact on road users. In this survey, we provide a comprehensive review of the literature on safety in traffic management systems. Specifically, we discuss the different safety issues that arise in traffic management systems, the current state of research on safety in these systems, and the techniques and methods proposed to ensure the safety of these systems. We also identify the limitations of the existing research and suggest future research directions.

* Designs 2023, 7, 100  
* Accepted by MDPI Designs journal, the Special Issue Design and Application of Intelligent Transportation Systems. 30 pages, 6 figures, published on 10 August 2023 
Viaarxiv icon

RegExplainer: Generating Explanations for Graph Neural Networks in Regression Task

Jul 25, 2023
Jiaxing Zhang, Zhuomin Chen, Hao Mei, Dongsheng Luo, Hua Wei

Figure 1 for RegExplainer: Generating Explanations for Graph Neural Networks in Regression Task
Figure 2 for RegExplainer: Generating Explanations for Graph Neural Networks in Regression Task
Figure 3 for RegExplainer: Generating Explanations for Graph Neural Networks in Regression Task
Figure 4 for RegExplainer: Generating Explanations for Graph Neural Networks in Regression Task

Graph regression is a fundamental task and has received increasing attention in a wide range of graph learning tasks. However, the inference process is often not interpretable. Most existing explanation techniques are limited to understanding GNN behaviors in classification tasks. In this work, we seek an explanation to interpret the graph regression models (XAIG-R). We show that existing methods overlook the distribution shifting and continuously ordered decision boundary, which hinders them away from being applied in the regression tasks. To address these challenges, we propose a novel objective based on the information bottleneck theory and introduce a new mix-up framework, which could support various GNNs in a model-agnostic manner. We further present a contrastive learning strategy to tackle the continuously ordered labels in regression task. To empirically verify the effectiveness of the proposed method, we introduce three benchmark datasets and a real-life dataset for evaluation. Extensive experiments show the effectiveness of the proposed method in interpreting GNN models in regression tasks.

Viaarxiv icon

Uncertainty-aware Grounded Action Transformation towards Sim-to-Real Transfer for Traffic Signal Control

Jul 23, 2023
Longchao Da, Hao Mei, Romir Sharma, Hua Wei

Figure 1 for Uncertainty-aware Grounded Action Transformation towards Sim-to-Real Transfer for Traffic Signal Control
Figure 2 for Uncertainty-aware Grounded Action Transformation towards Sim-to-Real Transfer for Traffic Signal Control
Figure 3 for Uncertainty-aware Grounded Action Transformation towards Sim-to-Real Transfer for Traffic Signal Control
Figure 4 for Uncertainty-aware Grounded Action Transformation towards Sim-to-Real Transfer for Traffic Signal Control

Traffic signal control (TSC) is a complex and important task that affects the daily lives of millions of people. Reinforcement Learning (RL) has shown promising results in optimizing traffic signal control, but current RL-based TSC methods are mainly trained in simulation and suffer from the performance gap between simulation and the real world. In this paper, we propose a simulation-to-real-world (sim-to-real) transfer approach called UGAT, which transfers a learned policy trained from a simulated environment to a real-world environment by dynamically transforming actions in the simulation with uncertainty to mitigate the domain gap of transition dynamics. We evaluate our method on a simulated traffic environment and show that it significantly improves the performance of the transferred RL policy in the real world.

* 8 pages, 3 figures 
Viaarxiv icon