Cooperative Hardware-Prompt Learning for Snapshot Compressive Imaging

Snapshot compressive imaging emerges as a promising technology for acquiring real-world hyperspectral signals. It uses an optical encoder and compressively produces the 2D measurement, followed by which the 3D hyperspectral data can be retrieved via training a deep reconstruction network. Existing reconstruction models are trained with a single hardware instance, whose performance is vulnerable to hardware perturbation or replacement, demonstrating an overfitting issue to the physical configuration. This defect limits the deployment of pre-trained models since they would suffer from large performance degradation when are assembled to unseen hardware. To better facilitate the reconstruction model with new hardware, previous efforts resort to centralized training by collecting multi-hardware and data, which is impractical when dealing with proprietary assets among institutions. In light of this, federated learning (FL) has become a feasible solution to enable cross-hardware cooperation without breaking privacy. However, the naive FedAvg is subject to client drift upon data heterogeneity owning to the hardware inconsistency. In this work, we tackle this challenge by marrying prompt tuning with FL to snapshot compressive imaging for the first time and propose an federated hardware-prompt learning (FedHP) method. Rather than mitigating the client drift by rectifying the gradients, which only takes effect on the learning manifold but fails to touch the heterogeneity rooted in the input data space, the proposed FedHP globally learns a hardware-conditioned prompter to align the data distribution, which serves as an indicator of the data inconsistency stemming from different pre-defined coded apertures. Extensive experiments demonstrate that the proposed method well coordinates the pre-trained model to indeterminate hardware configurations.

Integrated Sensing and Communication Complex CNN CSI Enhancer for 6G Networks

In this paper, we propose a novel integrated sensing and communication (ISAC) complex convolution neural network (CNN) CSI enhancer for 6G networks, which exploits the correlation between the sensing parameters, such as angle-of-arrival (AoA) and range, and the channel state information (CSI) to significantly improve the CSI estimation accuracy and further enhance the sensing accuracy. The ISAC complex CNN CSI enhancer uses the complex-value computation layers to form the CNN to better maintain the phase information of CSI. Furthermore, we incorporate the ISAC transform modules into the CNN enhancer to transform the CSI into the sparse angle-delay domain, which can be treated as images with prominent peaks and are suitable to be processed by CNN. Then, we further propose a novel biased FFT-based sensing scheme, where we actively add known phase bias terms to the original CSI to generate multiple estimation results using a simple FFT-based sensing method, and we finally calculate the average of all the debiased sensing results to obtain more accurate range estimates. The extensive simulation results show that the ISAC complex CNN CSI enhancer can converge within 30 training epochs. Its CSI estimation normalized mean square error (NMSE) is about 17 dB lower than the MMSE method, and the bit error rate (BER) of demodulation using the enhanced CSI approaches the perfect CSI. Finally, the range estimation MSE of the proposed biased FFT-based sensing method can approach the subspace-based method with much lower complexity.

Hierarchical Integration Diffusion Model for Realistic Image Deblurring

Diffusion models (DMs) have recently been introduced in image deblurring and exhibited promising performance, particularly in terms of details reconstruction. However, the diffusion model requires a large number of inference iterations to recover the clean image from pure Gaussian noise, which consumes massive computational resources. Moreover, the distribution synthesized by the diffusion model is often misaligned with the target results, leading to restrictions in distortion-based metrics. To address the above issues, we propose the Hierarchical Integration Diffusion Model (HI-Diff), for realistic image deblurring. Specifically, we perform the DM in a highly compacted latent space to generate the prior feature for the deblurring process. The deblurring process is implemented by a regression-based method to obtain better distortion accuracy. Meanwhile, the highly compact latent space ensures the efficiency of the DM. Furthermore, we design the hierarchical integration module to fuse the prior into the regression-based model from multiple scales, enabling better generalization in complex blurry scenarios. Comprehensive experiments on synthetic and real-world blur datasets demonstrate that our HI-Diff outperforms state-of-the-art methods. Code and trained models are available at https://github.com/zhengchen1999/HI-Diff.

EfficientSCI: Densely Connected Network with Space-time Factorization for Large-scale Video Snapshot Compressive Imaging

Video snapshot compressive imaging (SCI) uses a two-dimensional detector to capture consecutive video frames during a single exposure time. Following this, an efficient reconstruction algorithm needs to be designed to reconstruct the desired video frames. Although recent deep learning-based state-of-the-art (SOTA) reconstruction algorithms have achieved good results in most tasks, they still face the following challenges due to excessive model complexity and GPU memory limitations: 1) these models need high computational cost, and 2) they are usually unable to reconstruct large-scale video frames at high compression ratios. To address these issues, we develop an efficient network for video SCI by using dense connections and space-time factorization mechanism within a single residual block, dubbed EfficientSCI. The EfficientSCI network can well establish spatial-temporal correlation by using convolution in the spatial domain and Transformer in the temporal domain, respectively. We are the first time to show that an UHD color video with high compression ratio can be reconstructed from a snapshot 2D measurement using a single end-to-end deep learning model with PSNR above 32 dB. Extensive results on both simulation and real data show that our method significantly outperforms all previous SOTA algorithms with better real-time performance. The code is at https://github.com/ucaswangls/EfficientSCI.git.

Binarized Spectral Compressive Imaging

Existing deep learning models for hyperspectral image (HSI) reconstruction achieve good performance but require powerful hardwares with enormous memory and computational resources. Consequently, these methods can hardly be deployed on resource-limited mobile devices. In this paper, we propose a novel method, Binarized Spectral-Redistribution Network (BiSRNet), for efficient and practical HSI restoration from compressed measurement in snapshot compressive imaging (SCI) systems. Firstly, we redesign a compact and easy-to-deploy base model to be binarized. Then we present the basic unit, Binarized Spectral-Redistribution Convolution (BiSR-Conv). BiSR-Conv can adaptively redistribute the HSI representations before binarizing activation and uses a scalable hyperbolic tangent function to closer approximate the Sign function in backpropagation. Based on our BiSR-Conv, we customize four binarized convolutional modules to address the dimension mismatch and propagate full-precision information throughout the whole network. Finally, our BiSRNet is derived by using the proposed techniques to binarize the base model. Comprehensive quantitative and qualitative experiments manifest that our proposed BiSRNet outperforms state-of-the-art binarization methods and achieves comparable performance with full-precision algorithms. Code and models will be released at https://github.com/caiyuanhao1998/BiSCI and https://github.com/caiyuanhao1998/MST

Learn to Unlearn: A Survey on Machine Unlearning

Machine Learning (ML) models contain private information, and implementing the right to be forgotten is a challenging privacy issue in many data applications. Machine unlearning has emerged as an alternative to remove sensitive data from a trained model, but completely retraining ML models is often not feasible. This survey provides a concise appraisal of Machine Unlearning techniques, encompassing both exact and approximate methods, probable attacks, and verification approaches. The survey compares the merits and limitations each method and evaluates their performance using the Deltagrad exact machine unlearning method. The survey also highlights challenges like the pressing need for a robust model for non-IID deletion to mitigate fairness issues. Overall, the survey provides a thorough synopsis of machine unlearning techniques and applications, noting future research directions in this evolving field. The survey aims to be a valuable resource for researchers and practitioners seeking to provide privacy and equity in ML systems.

New Adversarial Image Detection Based on Sentiment Analysis

Deep Neural Networks (DNNs) are vulnerable to adversarial examples, while adversarial attack models, e.g., DeepFool, are on the rise and outrunning adversarial example detection techniques. This paper presents a new adversarial example detector that outperforms state-of-the-art detectors in identifying the latest adversarial attacks on image datasets. Specifically, we propose to use sentiment analysis for adversarial example detection, qualified by the progressively manifesting impact of an adversarial perturbation on the hidden-layer feature maps of a DNN under attack. Accordingly, we design a modularized embedding layer with the minimum learnable parameters to embed the hidden-layer feature maps into word vectors and assemble sentences ready for sentiment analysis. Extensive experiments demonstrate that the new detector consistently surpasses the state-of-the-art detection algorithms in detecting the latest attacks launched against ResNet and Inception neutral networks on the CIFAR-10, CIFAR-100 and SVHN datasets. The detector only has about 2 million parameters, and takes shorter than 4.6 milliseconds to detect an adversarial example generated by the latest attack models using a Tesla K80 GPU card.

Code-Division OFDM Joint Communication and Sensing System for 6G Machine-type Communication

The joint communication and sensing (JCS) system can provide higher spectrum efficiency and load-saving for 6G machine-type communication (MTC) applications by merging necessary communication and sensing abilities with unified spectrum and transceivers. In order to suppress the mutual interference between the communication and radar sensing signals to improve the communication reliability and radar sensing accuracy, we propose a novel code-division orthogonal frequency division multiplex (CD-OFDM) JCS MTC system, where MTC users can simultaneously and continuously conduct communication and sensing with each other. {\color{black} We propose a novel CD-OFDM JCS signal and corresponding successive-interference-cancellation (SIC) based signal processing technique that obtains code-division multiplex (CDM) gain, which is compatible with the prevalent orthogonal frequency division multiplex (OFDM) communication system.} To model the unified JCS signal transmission and reception process, we propose a novel unified JCS channel model. Finally, the simulation and numerical results are shown to verify the feasibility of the CD-OFDM JCS MTC system {\color{black} and the error propagation performance}. We show that the CD-OFDM JCS MTC system can achieve not only more reliable communication but also comparably robust radar sensing compared with the precedent OFDM JCS system, especially in low signal-to-interference-and-noise ratio (SINR) regime.

Sensing Performance of Cooperative Joint Sensing-Communication UAV Network

We propose a novel cooperative joint sensing-communication (JSC) unmanned aerial vehicle (UAV) network that can achieve downward-looking detection and transmit detection data simultaneously using the same time and frequency resources by exploiting the beam sharing scheme. The UAV network consists of a UAV that works as a fusion center (FCUAV) and multiple subordinate UAVs (SU). All UAVs fly at the fixed height. FCUAV integrates the sensing data of network and carries out downward-looking detection. SUs carry out downward-looking detection and transmit the sensing data to FCUAV. To achieve the beam sharing scheme, each UAV is equipped with a novel JSC antenna array that is composed of both the sensing subarray (SenA) and the communication subarray (ComA) in order to generate the sensing beam (SenB) and the communication beam (ComB) for detection and communication, respectively. SenB and ComB of each UAV share a total amount of radio power. Because of the spatial orthogonality of communication and sensing, SenB and ComB can be easily formed orthogonally. The upper bound of average cooperative sensing area (UB-ACSA) is defined as the metric to measure the sensing performance, which is related to the mutual sensing interference and the communication capacity. Numerical simulations prove the validity of the theoretical expressions for UB-ACSA of the network. The optimal number of UAVs and the optimal SenB power are identified under the total power constraint.

Adversarial Attacks and Defenses in Machine Learning-Powered Networks: A Contemporary Survey

Adversarial attacks and defenses in machine learning and deep neural network have been gaining significant attention due to the rapidly growing applications of deep learning in the Internet and relevant scenarios. This survey provides a comprehensive overview of the recent advancements in the field of adversarial attack and defense techniques, with a focus on deep neural network-based classification models. Specifically, we conduct a comprehensive classification of recent adversarial attack methods and state-of-the-art adversarial defense techniques based on attack principles, and present them in visually appealing tables and tree diagrams. This is based on a rigorous evaluation of the existing works, including an analysis of their strengths and limitations. We also categorize the methods into counter-attack detection and robustness enhancement, with a specific focus on regularization-based methods for enhancing robustness. New avenues of attack are also explored, including search-based, decision-based, drop-based, and physical-world attacks, and a hierarchical classification of the latest defense methods is provided, highlighting the challenges of balancing training costs with performance, maintaining clean accuracy, overcoming the effect of gradient masking, and ensuring method transferability. At last, the lessons learned and open challenges are summarized with future research opportunities recommended.