Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Models

Large Language Models (LLMs) have become a cornerstone in the field of Natural Language Processing (NLP), offering transformative capabilities in understanding and generating human-like text. However, with their rising prominence, the security and vulnerability aspects of these models have garnered significant attention. This paper presents a comprehensive survey of the various forms of attacks targeting LLMs, discussing the nature and mechanisms of these attacks, their potential impacts, and current defense strategies. We delve into topics such as adversarial attacks that aim to manipulate model outputs, data poisoning that affects model training, and privacy concerns related to training data exploitation. The paper also explores the effectiveness of different attack methodologies, the resilience of LLMs against these attacks, and the implications for model integrity and user trust. By examining the latest research, we provide insights into the current landscape of LLM vulnerabilities and defense mechanisms. Our objective is to offer a nuanced understanding of LLM attacks, foster awareness within the AI community, and inspire robust solutions to mitigate these risks in future developments.

Towards Explainable LiDAR Point Cloud Semantic Segmentation via Gradient Based Target Localization

Semantic Segmentation (SS) of LiDAR point clouds is essential for many applications, such as urban planning and autonomous driving. While much progress has been made in interpreting SS predictions for images, interpreting point cloud SS predictions remains a challenge. This paper introduces pGS-CAM, a novel gradient-based method for generating saliency maps in neural network activation layers. Inspired by Grad-CAM, which uses gradients to highlight local importance, pGS-CAM is robust and effective on a variety of datasets (SemanticKITTI, Paris-Lille3D, DALES) and 3D deep learning architectures (KPConv, RandLANet). Our experiments show that pGS-CAM effectively accentuates the feature learning in intermediate activations of SS architectures by highlighting the contribution of each point. This allows us to better understand how SS models make their predictions and identify potential areas for improvement. Relevant codes are available at https://github.com/geoai4cities/pGS-CAM.

A Survey on Integrated Sensing and Communication with Intelligent Metasurfaces: Trends, Challenges, and Opportunities

The emergence of various technologies demanding both high data rates and precise sensing performance, such as autonomous vehicles and internet of things devices, has propelled an increasing popularity of integrated sensing and communication (ISAC) in recent years. ISAC offers an efficient framework for communication and sensing where both functionalities are carried out in a shared spectrum, utilizing the same hardware, beamformer and waveform design. At the same time, intelligent metasurfaces have been identified as an architectural enabler for the upcoming sixth-generation (6G) of wireless communication due to their ability to control the propagation environment in an energy-efficient manner. Due to the potential of metasurfaces to enhance both communication and sensing performance, numerous papers have explored the performance gains of using metasurfaces to improve ISAC. This survey reviews the existing literature on metasurface-assisted ISAC, detailing the associated challenges and opportunities. To provide a comprehensive overview, we commence by offering relevant background information on standalone metasurface-assisted communication and metasurface-assisted sensing systems, followed by a discussion on the fundamentals of ISAC. The core part of the paper then summarizes the state-of-the-art studies on metasurface-assisted ISAC with metasurfaces employed as separate entities placed between the transmitter and receiver, also known as reconfigurable intelligent surfaces, with an emphasis on its two levels of integration: radio-communications co-existence and dual-function radar-communications. We also review the current works in the area of holographic ISAC where metasurfaces are used to form part of ISAC transmitter. Within each category, the challenges, opportunities and future research directions are also highlighted.

X-Eval: Generalizable Multi-aspect Text Evaluation via Augmented Instruction Tuning with Auxiliary Evaluation Aspects

Natural Language Generation (NLG) typically involves evaluating the generated text in various aspects (e.g., consistency and naturalness) to obtain a comprehensive assessment. However, multi-aspect evaluation remains challenging as it may require the evaluator to generalize to any given evaluation aspect even if it's absent during training. In this paper, we introduce X-Eval, a two-stage instruction tuning framework to evaluate the text in both seen and unseen aspects customized by end users. X-Eval consists of two learning stages: the vanilla instruction tuning stage that improves the model's ability to follow evaluation instructions, and an enhanced instruction tuning stage that exploits the connections between fine-grained evaluation aspects to better assess text quality. To support the training of X-Eval, we collect AspectInstruct, the first instruction tuning dataset tailored for multi-aspect NLG evaluation spanning 27 diverse evaluation aspects with 65 tasks. To enhance task diversity, we devise an augmentation strategy that converts human rating annotations into diverse forms of NLG evaluation tasks, including scoring, comparison, ranking, and Boolean question answering. Extensive experiments across three essential categories of NLG tasks: dialogue generation, summarization, and data-to-text coupled with 21 aspects in meta-evaluation, demonstrate that our X-Eval enables even a lightweight language model to achieve a comparable if not higher correlation with human judgments compared to the state-of-the-art NLG evaluators, such as GPT-4.

pCTFusion: Point Convolution-Transformer Fusion with Semantic Aware Loss for Outdoor LiDAR Point Cloud Segmentation

LiDAR-generated point clouds are crucial for perceiving outdoor environments. The segmentation of point clouds is also essential for many applications. Previous research has focused on using self-attention and convolution (local attention) mechanisms individually in semantic segmentation architectures. However, there is limited work on combining the learned representations of these attention mechanisms to improve performance. Additionally, existing research that combines convolution with self-attention relies on global attention, which is not practical for processing large point clouds. To address these challenges, this study proposes a new architecture, pCTFusion, which combines kernel-based convolutions and self-attention mechanisms for better feature learning and capturing local and global dependencies in segmentation. The proposed architecture employs two types of self-attention mechanisms, local and global, based on the hierarchical positions of the encoder blocks. Furthermore, the existing loss functions do not consider the semantic and position-wise importance of the points, resulting in reduced accuracy, particularly at sharp class boundaries. To overcome this, the study models a novel attention-based loss function called Pointwise Geometric Anisotropy (PGA), which assigns weights based on the semantic distribution of points in a neighborhood. The proposed architecture is evaluated on SemanticKITTI outdoor dataset and showed a 5-7% improvement in performance compared to the state-of-the-art architectures. The results are particularly encouraging for minor classes, often misclassified due to class imbalance, lack of space, and neighbor-aware feature encoding. These developed methods can be leveraged for the segmentation of complex datasets and can drive real-world applications of LiDAR point cloud.

Robust Beamforming Design for Energy Efficiency and Fairness Maximization in RIS-Assisted mmWave Communications

The use of reconfigurable intelligent surfaces (RISs) has been proposed in the past few years to achieve a better communication system performance by creating a programmable wireless propagation environment. In this paper, we target maximizing both energy efficiency and user fairness in RIS-assisted millimeter-wave systems with imperfect channel state information. We formulate the energy efficiency and fairness maximization problem as a multi-objective optimization problem. We split the corresponding multi-objective optimization problem into two stages using a lexicographic approach. In the first stage, the energy efficiency is maximized; then in the second stage, the fairness is maximized subject to a maximum reduction in the optimal value of the energy efficiency. We propose a projected gradient ascent based alternating optimization procedure to solve the optimization problem in each stage. We further employ the penalty dual decomposition method to address the challenging energy efficiency constraint in the second stage. Simulation results show that the proposed algorithm can achieve a better trade-off between energy efficiency and fairness compared to the methods that target only one of those metrics.

Controlled Text Generation with Hidden Representation Transformations

We propose CHRT (Control Hidden Representation Transformation) - a controlled language generation framework that steers large language models to generate text pertaining to certain attributes (such as toxicity). CHRT gains attribute control by modifying the hidden representation of the base model through learned transformations. We employ a contrastive-learning framework to learn these transformations that can be combined to gain multi-attribute control. The effectiveness of CHRT is experimentally shown by comparing it with seven baselines over three attributes. CHRT outperforms all the baselines in the task of detoxification, positive sentiment steering, and text simplification while minimizing the loss in linguistic qualities. Further, our approach has the lowest inference latency of only 0.01 seconds more than the base model, making it the most suitable for high-performance production environments. We open-source our code and release two novel datasets to further propel controlled language generation research.

SCA-Based Beamforming Optimization for IRS-Enabled Secure Integrated Sensing and Communication

Integrated sensing and communication (ISAC) is expected to be offered as a fundamental service in the upcoming sixth-generation (6G) communications standard. However, due to the exposure of information-bearing signals to the sensing targets, ISAC poses unique security challenges. In recent years, intelligent reflecting surfaces (IRSs) have emerged as a novel hardware technology capable of enhancing the physical layer security of wireless communication systems. Therefore, in this paper, we consider the problem of transmit and reflective beamforming design in a secure IRS-enabled ISAC system to maximize the beampattern gain at the target. The formulated non-convex optimization problem is challenging to solve due to the intricate coupling between the design variables. Moreover, alternating optimization (AO) based methods are inefficient in finding a solution in such scenarios, and convergence to a stationary point is not theoretically guaranteed. Therefore, we propose a novel successive convex approximation (SCA)-based second-order cone programming (SOCP) scheme in which all of the design variables are updated simultaneously in each iteration. The proposed SCA-based method significantly outperforms a penalty-based benchmark scheme previously proposed in this context. Moreover, we also present a detailed complexity analysis of the proposed scheme, and show that despite having slightly higher per-iteration complexity than the benchmark approach the average problem-solving time of the proposed method is notably lower than that of the benchmark scheme.

A Low-Complexity Solution to Sum Rate Maximization for IRS-assisted SWIPT-MIMO Broadcasting

This paper focuses on the fundamental problem of maximizing the achievable weighted sum rate (WSR) at information receivers (IRs) in an intelligent reflecting surface (IRS) assisted simultaneous wireless information and power transfer system under a multiple-input multiple-output (SWIPT-MIMO) setting, subject to a quality-of-service (QoS) constraint at the energy receivers (ERs). Notably, due to the coupling between the transmit precoding matrix and the passive beamforming vector in the QoS constraint, the formulated non-convex optimization problem is challenging to solve. We first decouple the design variables in the constraints following a penalty dual decomposition method, and then apply an alternating gradient projection algorithm to achieve a stationary solution to the reformulated optimization problem. The proposed algorithm nearly doubles the WSR compared to that achieved by a block-coordinate descent (BCD) based benchmark scheme. At the same time, the complexity of the proposed scheme grows linearly with the number of IRS elements while that of the benchmark scheme is proportional to the cube of the number of IRS elements.

On Energy Efficiency and Fairness Maximization in RIS-Assisted MU-MISO mmWave Communications

Reconfigurable intelligent surfaces (RISs) are considered to be a promising solution to overcome the blockage issue in the millimeter-wave (mmWave) band. Energy efficiency is an important performance metric in mmWave systems with a large number of antennas. However, due to the severe path loss in mmWave systems, resource allocation algorithms tend to allocate most of the resources for the benefit of the users with higher channel gains. In this paper, we propose a lexicographic-based approach to find the optimal power allocation, RIS phase shift matrix, and analog precoders that maximize both energy efficiency and user fairness. We solve the corresponding multi-objective optimization problem in two stages. In the first stage, we maximize the energy efficiency, and in the second stage we maximize the fairness subject to a minimum energy efficiency constraint. We propose an alternating optimization procedure to solve the optimization problem in each stage. The optimal power allocation is found using Dinkelbach's method in the first stage and using convex optimization techniques in the second stage, the RIS phase shift matrix is found using a gradient ascent algorithm, and the analog precoder is determined using beam alignment. Numerical results show that the proposed algorithm can achieve an excellent trade-off between the energy efficiency and fairness by boosting the minimum weighted rate with a minor and controllable reduction in the energy efficiency.