Difference-based Deep Convolutional Neural Network for Simulation-to-reality UAV Fault Diagnosis

Identifying the fault in propellers is important to keep quadrotors operating safely and efficiently. The simulation-to-reality (sim-to-real) UAV fault diagnosis methods provide a cost-effective and safe approach to detect the propeller faults. However, due to the gap between simulation and reality, classifiers trained with simulated data usually underperform in real flights. In this work, a new deep neural network (DNN) model is presented to address the above issue. It uses the difference features extracted by deep convolutional neural networks (DDCNN) to reduce the sim-to-real gap. Moreover, a new domain adaptation method is presented to further bring the distribution of the real-flight data closer to that of the simulation data. The experimental results show that the proposed approach can achieve an accuracy of 97.9\% in detecting propeller faults in real flight. Feature visualization was performed to help better understand our DDCNN model.

RIS-Position and Orientation Estimation in Dispersive mmWave MIMO Scenarios

In this paper, we investigate the problem of estimating the position and the angle of rotation of a mobile station (MS) in a millimeter wave (mmWave) multiple-input-multiple-output (MIMO) system aided by a reconfigurable intelligent surface (RIS). The virtual line-of-sight (VLoS) link created by the RIS and the non-line-of-sight (NLoS) links that originate from scatterers in the considered environment are utilized to facilitate the estimation. A two-step positioning scheme is exploited, where the channel parameters are first acquired, and the position-related parameters are then estimated. The channel parameters are obtained through a coarser and a subsequent finer estimation processes. As for the coarse estimation, the distributed compressed sensing orthogonal simultaneous matching pursuit (DCS-SOMP) algorithm, the maximum likelihood (ML) algorithm, and the discrete Fourier transform (DFT) are utilized to separately estimate the channel parameters. The obtained channel parameters are then jointly refined by using the space-alternating generalized expectation maximization (SAGE) algorithm, which circumvents the high-dimensional optimization issue of ML estimation. Departing from the estimated channel parameters, the positioning-related parameters are estimated. The performance of estimating the channel-related and position-related parameters is theoretically quantified by using the Cramer-Rao lower bound (CRLB). Simulation results demonstrate the superior performance of the proposed positioning algorithms.

Simulation-to-reality UAV Fault Diagnosis with Deep Learning

Accurate diagnosis of propeller faults is crucial for ensuring the safe and efficient operation of quadrotors. Training a fault classifier using simulated data and deploying it on a real quadrotor is a cost-effective and safe approach. However, the simulation-to-reality gap often leads to poor performance of the classifier when applied in real flight. In this work, we propose a deep learning model that addresses this issue by utilizing newly identified features (NIF) as input and utilizing domain adaptation techniques to reduce the simulation-to-reality gap. In addition, we introduce an adjusted simulation model that generates training data that more accurately reflects the behavior of real quadrotors. The experimental results demonstrate that our proposed approach achieves an accuracy of 96\% in detecting propeller faults. To the best of our knowledge, this is the first reliable and efficient method for simulation-to-reality fault diagnosis of quadrotor propellers.

Anchor-Based Adversarially Robust Zero-Shot Learning Driven by Language

Deep neural networks are vulnerable to adversarial attacks. We consider adversarial defense in the case of zero-shot image classification setting, which has rarely been explored because both adversarial defense and zero-shot learning are challenging. We propose LAAT, a novel Language-driven, Anchor-based Adversarial Training strategy, to improve the adversarial robustness in a zero-shot setting. LAAT uses a text encoder to obtain fixed anchors (normalized feature embeddings) of each category, then uses these anchors to perform adversarial training. The text encoder has the property that semantically similar categories can be mapped to neighboring anchors in the feature space. By leveraging this property, LAAT can make the image model adversarially robust on novel categories without any extra examples. Experimental results show that our method achieves impressive zero-shot adversarial performance, even surpassing the previous state-of-the-art adversarially robust one-shot methods in most attacking settings. When models are trained with LAAT on large datasets like ImageNet-1K, they can have substantial zero-shot adversarial robustness across several downstream datasets.

Deep-learning-based on-chip rapid spectral imaging with high spatial resolution

Spectral imaging extends the concept of traditional color cameras to capture images across multiple spectral channels and has broad application prospects. Conventional spectral cameras based on scanning methods suffer from low acquisition speed and large volume. On-chip computational spectral imaging based on metasurface filters provides a promising scheme for portable applications, but endures long computation time for point-by-point iterative spectral reconstruction and mosaic effect in the reconstructed spectral images. In this study, we demonstrated on-chip rapid spectral imaging eliminating the mosaic effect in the spectral image by deep-learning-based spectral data cube reconstruction. We experimentally achieved four orders of magnitude speed improvement than iterative spectral reconstruction and high fidelity of spectral reconstruction over 99% for a standard color board. In particular, we demonstrated video-rate spectral imaging for moving objects and outdoor driving scenes with good performance for recognizing metamerism, where the concolorous sky and white cars can be distinguished via their spectra, showing great potential for autonomous driving and other practical applications in the field of intelligent perception.

EPR-Net: Constructing non-equilibrium potential landscape via a variational force projection formulation

We present a novel yet simple deep learning approach, dubbed EPR-Net, for constructing the potential landscape of high-dimensional non-equilibrium steady state (NESS) systems. The key idea of our approach is to utilize the fact that the negative potential gradient is the orthogonal projection of the driving force in a weighted Hilbert space with respect to the steady-state distribution. The constructed loss function also coincides with the entropy production rate (EPR) formula in NESS theory. This approach can be extended to dealing with dimensionality reduction and state-dependent diffusion coefficients in a unified fashion. The robustness and effectiveness of the proposed approach are demonstrated by numerical studies of several high-dimensional biophysical models with multi-stability, limit cycle, or strange attractor with non-vanishing noise.

Machine Learning for Large-Scale Optimization in 6G Wireless Networks

The sixth generation (6G) wireless systems are envisioned to enable the paradigm shift from "connected things" to "connected intelligence", featured by ultra high density, large-scale, dynamic heterogeneity, diversified functional requirements and machine learning capabilities, which leads to a growing need for highly efficient intelligent algorithms. The classic optimization-based algorithms usually require highly precise mathematical model of data links and suffer from poor performance with high computational cost in realistic 6G applications. Based on domain knowledge (e.g., optimization models and theoretical tools), machine learning (ML) stands out as a promising and viable methodology for many complex large-scale optimization problems in 6G, due to its superior performance, generalizability, computational efficiency and robustness. In this paper, we systematically review the most representative "learning to optimize" techniques in diverse domains of 6G wireless networks by identifying the inherent feature of the underlying optimization problem and investigating the specifically designed ML frameworks from the perspective of optimization. In particular, we will cover algorithm unrolling, learning to branch-and-bound, graph neural network for structured optimization, deep reinforcement learning for stochastic optimization, end-to-end learning for semantic optimization, as well as federated learning for distributed optimization, for solving challenging large-scale optimization problems arising from various important wireless applications. Through the in-depth discussion, we shed light on the excellent performance of ML-based optimization algorithms with respect to the classical methods, and provide insightful guidance to develop advanced ML techniques in 6G networks.

A Deep Learning Method for Real-time Bias Correction of Wind Field Forecasts in the Western North Pacific

Forecasts by the European Centre for Medium-Range Weather Forecasts (ECMWF; EC for short) can provide a basis for the establishment of maritime-disaster warning systems, but they contain some systematic biases.The fifth-generation EC atmospheric reanalysis (ERA5) data have high accuracy, but are delayed by about 5 days. To overcome this issue, a spatiotemporal deep-learning method could be used for nonlinear mapping between EC and ERA5 data, which would improve the quality of EC wind forecast data in real time. In this study, we developed the Multi-Task-Double Encoder Trajectory Gated Recurrent Unit (MT-DETrajGRU) model, which uses an improved double-encoder forecaster architecture to model the spatiotemporal sequence of the U and V components of the wind field; we designed a multi-task learning loss function to correct wind speed and wind direction simultaneously using only one model. The study area was the western North Pacific (WNP), and real-time rolling bias corrections were made for 10-day wind-field forecasts released by the EC between December 2020 and November 2021, divided into four seasons. Compared with the original EC forecasts, after correction using the MT-DETrajGRU model the wind speed and wind direction biases in the four seasons were reduced by 8-11% and 9-14%, respectively. In addition, the proposed method modelled the data uniformly under different weather conditions. The correction performance under normal and typhoon conditions was comparable, indicating that the data-driven mode constructed here is robust and generalizable.

Circular Accessible Depth: A Robust Traversability Representation for UGV Navigation

In this paper, we present the Circular Accessible Depth (CAD), a robust traversability representation for an unmanned ground vehicle (UGV) to learn traversability in various scenarios containing irregular obstacles. To predict CAD, we propose a neural network, namely CADNet, with an attention-based multi-frame point cloud fusion module, Stability-Attention Module (SAM), to encode the spatial features from point clouds captured by LiDAR. CAD is designed based on the polar coordinate system and focuses on predicting the border of traversable area. Since it encodes the spatial information of the surrounding environment, which enables a semi-supervised learning for the CADNet, and thus desirably avoids annotating a large amount of data. Extensive experiments demonstrate that CAD outperforms baselines in terms of robustness and precision. We also implement our method on a real UGV and show that it performs well in real-world scenarios.

Semantic optical fiber communication system

The current optical communication systems minimize bit or symbol errors without considering the semantic meaning behind digital bits, thus transmitting a lot of unnecessary information. We propose and experimentally demonstrate a semantic optical fiber communication (SOFC) system. Instead of encoding information into bits for transmission, semantic information is extracted from the source using deep learning. The generated semantic symbols are then directly transmitted through an optical fiber. Compared with the bit-based structure, the SOFC system achieved higher information compression and a more stable performance, especially in the low received optical power regime, and enhanced the robustness against optical link impairments. This work introduces an intelligent optical communication system at the human analytical thinking level, which is a significant step toward a breakthrough in the current optical communication architecture.