VAR: Visual Analysis for Rashomon Set of Machine Learning Models' Performance

Evaluating the performance of closely matched machine learning(ML) models under specific conditions has long been a focus of researchers in the field of machine learning. The Rashomon set is a collection of closely matched ML models, encompassing a wide range of models with similar accuracies but different structures. Traditionally, the analysis of these sets has focused on vertical structural analysis, which involves comparing the corresponding features at various levels within the ML models. However, there has been a lack of effective visualization methods for horizontally comparing multiple models with specific features. We propose the VAR visualization solution. VAR uses visualization to perform comparisons of ML models within the Rashomon set. This solution combines heatmaps and scatter plots to facilitate the comparison. With the help of VAR, ML model developers can identify the optimal model under specific conditions and better understand the Rashomon set's overall characteristics.

TA3: Testing Against Adversarial Attacks on Machine Learning Models

Adversarial attacks are major threats to the deployment of machine learning (ML) models in many applications. Testing ML models against such attacks is becoming an essential step for evaluating and improving ML models. In this paper, we report the design and development of an interactive system for aiding the workflow of Testing Against Adversarial Attacks (TA3). In particular, with TA3, human-in-the-loop (HITL) enables human-steered attack simulation and visualization-assisted attack impact evaluation. While the current version of TA3 focuses on testing decision tree models against adversarial attacks based on the One Pixel Attack Method, it demonstrates the importance of HITL in ML testing and the potential application of HITL to the ML testing workflows for other types of ML models and other types of adversarial attacks.

iGAiVA: Integrated Generative AI and Visual Analytics in a Machine Learning Workflow for Text Classification

In developing machine learning (ML) models for text classification, one common challenge is that the collected data is often not ideally distributed, especially when new classes are introduced in response to changes of data and tasks. In this paper, we present a solution for using visual analytics (VA) to guide the generation of synthetic data using large language models. As VA enables model developers to identify data-related deficiency, data synthesis can be targeted to address such deficiency. We discuss different types of data deficiency, describe different VA techniques for supporting their identification, and demonstrate the effectiveness of targeted data synthesis in improving model accuracy. In addition, we present a software tool, iGAiVA, which maps four groups of ML tasks into four VA views, integrating generative AI and VA into an ML workflow for developing and improving text classification models.

The Importance of Autonomous Driving Using 5G Technology

The three keys to autonomous driving are sensors, data integration, and 100% safety decisions. In the past, due to the high latency and low reliability of the network, many decisions had to be made locally in the vehicle. This puts high demands on the vehicle itself, which results in the dilatory commercialization of automatic driving. With the advent of 5G, these situations will be greatly improved. In this paper, we present the improvements that 5G technology brings to autonomous vehicles especially in terms of latency and reliability amongst the multitude of other factors. The paper analyzes the specific areas where 5G can improve for autonomous vehicles and Intelligent Transport Systems in general (ITS) and looks forward to the application of 5G technology in the future.