Spiking Semantic Communication for Feature Transmission with HARQ

In Collaborative Intelligence (CI), the Artificial Intelligence (AI) model is divided between the edge and the cloud, with intermediate features being sent from the edge to the cloud for inference. Several deep learning-based Semantic Communication (SC) models have been proposed to reduce feature transmission overhead and mitigate channel noise interference. Previous research has demonstrated that Spiking Neural Network (SNN)-based SC models exhibit greater robustness on digital channels compared to Deep Neural Network (DNN)-based SC models. However, the existing SNN-based SC models require fixed time steps, resulting in fixed transmission bandwidths that cannot be adaptively adjusted based on channel conditions. To address this issue, this paper introduces a novel SC model called SNN-SC-HARQ, which combines the SNN-based SC model with the Hybrid Automatic Repeat Request (HARQ) mechanism. SNN-SC-HARQ comprises an SNN-based SC model that supports the transmission of features at varying bandwidths, along with a policy model that determines the appropriate bandwidth. Experimental results show that SNN-SC-HARQ can dynamically adjust the bandwidth according to the channel conditions without performance loss.

S-JSCC: A Digital Joint Source-Channel Coding Framework based on Spiking Neural Network

Nowadays, deep learning-based joint source-channel coding (JSCC) is getting attention, and it shows excellent performance compared with separate source and channel coding (SSCC). However, most JSCC works are only designed, trained, and tested on additive white Gaussian noise (AWGN) channels to transmit analog signals. In current communication systems, digital signals are considered more. Hence, it is necessary to design an end-to-end JSCC framework for digital signal transmission. In this paper, we propose a digital JSCC framework (S-JSCC) based on spiking neural network (SNN) to tackle this problem. The SNN is used to compress the feature of the deep model, and the compressed results are transmitted over digital channels such as binary symmetric channel (BSC) and binary erasure channel (BEC). Since the outputs of SNN are binary spikes, the framework can be applied directly to digital channels without extra quantization. Moreover, we propose a new spiking neuron and regularization method to improve the performance and robustness of the system. The experimental results show that under digital channels, the proposed S-JSCC framework performs better than the state-of-the-art convolution neural network (CNN)-based JSCC framework, which needs extra quantization.

Constellation Design for Deep Joint Source-Channel Coding

Deep learning-based joint source-channel coding (JSCC) has shown excellent performance in image and feature transmission. However, the output values of the JSCC encoder are continuous, which makes the constellation of modulation complex and dense. It is hard and expensive to design radio frequency chains for transmitting such full-resolution constellation points. In this paper, two methods of mapping the full-resolution constellation to finite constellation are proposed for real system implementation. The constellation mapping results of the proposed methods correspond to regular constellation and irregular constellation, respectively. We apply the methods to existing deep JSCC models and evaluate them on AWGN channels with different signal-to-noise ratios (SNRs). Experimental results show that the proposed methods outperform the traditional uniform quadrature amplitude modulation (QAM) constellation mapping method by only adding a few additional parameters.

Deep Joint Source-Channel Coding for Multi-Task Network

Multi-task learning (MTL) is an efficient way to improve the performance of related tasks by sharing knowledge. However, most existing MTL networks run on a single end and are not suitable for collaborative intelligence (CI) scenarios. In this work, we propose an MTL network with a deep joint source-channel coding (JSCC) framework, which allows operating under CI scenarios. We first propose a feature fusion based MTL network (FFMNet) for joint object detection and semantic segmentation. Compared with other MTL networks, FFMNet gets higher performance with fewer parameters. Then FFMNet is split into two parts, which run on a mobile device and an edge server respectively. The feature generated by the mobile device is transmitted through the wireless channel to the edge server. To reduce the transmission overhead of the intermediate feature, a deep JSCC network is designed. By combining two networks together, the whole model achieves 512x compression for the intermediate feature and a performance loss within 2% on both tasks. At last, by training with noise, the FFMNet with JSCC is robust to various channel conditions and outperforms the separate source and channel coding scheme.