Ultra-low Precision Multiplication-free Training for Deep Neural Networks

The training for deep neural networks (DNNs) demands immense energy consumption, which restricts the development of deep learning as well as increases carbon emissions. Thus, the study of energy-efficient training for DNNs is essential. In training, the linear layers consume the most energy because of the intense use of energy-consuming full-precision (FP32) multiplication in multiply-accumulate (MAC). The energy-efficient works try to decrease the precision of multiplication or replace the multiplication with energy-efficient operations such as addition or bitwise shift, to reduce the energy consumption of FP32 multiplications. However, the existing energy-efficient works cannot replace all of the FP32 multiplications during both forward and backward propagation with low-precision energy-efficient operations. In this work, we propose an Adaptive Layer-wise Scaling PoT Quantization (ALS-POTQ) method and a Multiplication-Free MAC (MF-MAC) to replace all of the FP32 multiplications with the INT4 additions and 1-bit XOR operations. In addition, we propose Weight Bias Correction and Parameterized Ratio Clipping techniques for stable training and improving accuracy. In our training scheme, all of the above methods do not introduce extra multiplications, so we reduce up to 95.8% of the energy consumption in linear layers during training. Experimentally, we achieve an accuracy degradation of less than 1% for CNN models on ImageNet and Transformer model on the WMT En-De task. In summary, we significantly outperform the existing methods for both energy efficiency and accuracy.

Dirichlet-based Uncertainty Calibration for Active Domain Adaptation

Active domain adaptation (DA) aims to maximally boost the model adaptation on a new target domain by actively selecting limited target data to annotate, whereas traditional active learning methods may be less effective since they do not consider the domain shift issue. Despite active DA methods address this by further proposing targetness to measure the representativeness of target domain characteristics, their predictive uncertainty is usually based on the prediction of deterministic models, which can easily be miscalibrated on data with distribution shift. Considering this, we propose a \textit{Dirichlet-based Uncertainty Calibration} (DUC) approach for active DA, which simultaneously achieves the mitigation of miscalibration and the selection of informative target samples. Specifically, we place a Dirichlet prior on the prediction and interpret the prediction as a distribution on the probability simplex, rather than a point estimate like deterministic models. This manner enables us to consider all possible predictions, mitigating the miscalibration of unilateral prediction. Then a two-round selection strategy based on different uncertainty origins is designed to select target samples that are both representative of target domain and conducive to discriminability. Extensive experiments on cross-domain image classification and semantic segmentation validate the superiority of DUC.

Reconfigurable Massive MIMO: Harnessing the Power of the Electromagnetic Domain for Enhanced Information Transfer

The capacity of commercial massive multiple-input multiple-output (mMIMO) systems is constrained by the limited array aperture at the base station, and cannot meet the ever-increasing traffic demands of wireless networks. Given the array aperture, holographic MIMO with infinitesimal antenna spacing can maximize the capacity, but is physically unrealizable. As a promising alternative, reconfigurable mMIMO is proposed to harness the unexploited power of the electromagnetic (EM) domain for enhanced information transfer. Specifically, the reconfigurable pixel antenna technology provides each antenna with an adjustable EM radiation (EMR) pattern, introducing extra degrees of freedom for information transfer in the EM domain. In this article, we present the concept and benefits of availing the EMR domain for mMIMO transmission. Moreover, we propose a viable architecture for reconfigurable mMIMO systems, and the associated system model and downlink precoding are also discussed. In particular, a three-level precoding scheme is proposed, and simulation results verify its considerable spectral and energy efficiency advantages compared to traditional mMIMO systems. Finally, we further discuss the challenges, insights, and prospects of deploying reconfigurable mMIMO, along with the associated hardware, algorithms, and fundamental theory.

Online Symbolic Regression with Informative Query

Symbolic regression, the task of extracting mathematical expressions from the observed data $\{ \vx_i, y_i \}$, plays a crucial role in scientific discovery. Despite the promising performance of existing methods, most of them conduct symbolic regression in an \textit{offline} setting. That is, they treat the observed data points as given ones that are simply sampled from uniform distributions without exploring the expressive potential of data. However, for real-world scientific problems, the data used for symbolic regression are usually actively obtained by doing experiments, which is an \textit{online} setting. Thus, how to obtain informative data that can facilitate the symbolic regression process is an important problem that remains challenging. In this paper, we propose QUOSR, a \textbf{qu}ery-based framework for \textbf{o}nline \textbf{s}ymbolic \textbf{r}egression that can automatically obtain informative data in an iterative manner. Specifically, at each step, QUOSR receives historical data points, generates new $\vx$, and then queries the symbolic expression to get the corresponding $y$, where the $(\vx, y)$ serves as new data points. This process repeats until the maximum number of query steps is reached. To make the generated data points informative, we implement the framework with a neural network and train it by maximizing the mutual information between generated data points and the target expression. Through comprehensive experiments, we show that QUOSR can facilitate modern symbolic regression methods by generating informative data.

A method for incremental discovery of financial event types based on anomaly detection

Event datasets in the financial domain are often constructed based on actual application scenarios, and their event types are weakly reusable due to scenario constraints; at the same time, the massive and diverse new financial big data cannot be limited to the event types defined for specific scenarios. This limitation of a small number of event types does not meet our research needs for more complex tasks such as the prediction of major financial events and the analysis of the ripple effects of financial events. In this paper, a three-stage approach is proposed to accomplish incremental discovery of event types. For an existing annotated financial event dataset, the three-stage approach consists of: for a set of financial event data with a mixture of original and unknown event types, a semi-supervised deep clustering model with anomaly detection is first applied to classify the data into normal and abnormal events, where abnormal events are events that do not belong to known types; then normal events are tagged with appropriate event types and abnormal events are reasonably clustered. Finally, a cluster keyword extraction method is used to recommend the type names of events for the new event clusters, thus incrementally discovering new event types. The proposed method is effective in the incremental discovery of new event types on real data sets.

Movable-Antenna Enhanced Multiuser Communication via Antenna Position Optimization

Movable antenna (MA) is a promising technology to improve wireless communication performance by varying the antenna position in a given finite area at the transceivers to create more favorable channel conditions. In this paper, we investigate the MA-enhanced multiple-access channel (MAC) for the uplink transmission from multiple users each equipped with a single MA to a base station (BS) with a fixed-position antenna (FPA) array. A field-response based channel model is used to characterize the multi-path channel between the antenna array of the BS and each user's MA with a flexible position. To evaluate the MAC performance gain provided by MAs, we formulate an optimization problem for minimizing the total transmit power of users, subject to a minimum-achievable-rate requirement for each user, where the positions of MAs and the transmit powers of users, as well as the receive combining matrix at the BS are jointly optimized. To solve this non-convex optimization problem involving intricately coupled variables, we develop two algorithms based on zero-forcing (ZF) and minimum mean square error (MMSE) combining methods, respectively. Specifically, for each algorithm, the combining matrix of the BS and the total transmit power of users are expressed as a function of the MAs' position vectors, which are then optimized by using the gradient descent method in an iterative manner. It is shown that the proposed ZF-based and MMSE-based algorithms can converge to high-quality suboptimal solutions with low computational complexities. Simulation results demonstrate that the proposed solutions for MA-enhanced multiple access systems can significantly decrease the total transmit power of users as compared to conventional FPA systems under both perfect and imperfect field-response information.

Intelligent Reflecting Surface Aided Wireless Sensing: Applications and Design Issues

Intelligent reflecting surface (IRS) is an emerging technology that is able to significantly improve the performance of wireless communications, by smartly tuning signal reflections at a large number of passive reflecting elements. On the other hand, with ubiquitous wireless devices and ambient radio-frequency signals, wireless sensing has become a promising new application for the next-generation/6G wireless networks. By synergizing low-cost IRS and fertile wireless sensing applications, this article proposes a new IRS-aided sensing paradigm for enhancing the performance of wireless sensing cost-effectively. First, we provide an overview of wireless sensing applications and the new opportunities of utilizing IRS for overcoming their performance limitations in practical scenarios. Next, we discuss IRS-aided sensing schemes based on three approaches, namely, passive sensing, semi-passive sensing, and active sensing. We compare their pros and cons in terms of performance, hardware cost and implementation complexity, and outline their main design issues including IRS deployment, channel acquisition and reflection design, as well as sensing algorithms. Finally, numerical results are presented to demonstrate the great potential of IRS for improving wireless sensing accuracy and the superior performance of IRS active sensing compared to other schemes.

Artificial Intelligence System for Detection and Screening of Cardiac Abnormalities using Electrocardiogram Images

The artificial intelligence (AI) system has achieved expert-level performance in electrocardiogram (ECG) signal analysis. However, in underdeveloped countries or regions where the healthcare information system is imperfect, only paper ECGs can be provided. Analysis of real-world ECG images (photos or scans of paper ECGs) remains challenging due to complex environments or interference. In this study, we present an AI system developed to detect and screen cardiac abnormalities (CAs) from real-world ECG images. The system was evaluated on a large dataset of 52,357 patients from multiple regions and populations across the world. On the detection task, the AI system obtained area under the receiver operating curve (AUC) of 0.996 (hold-out test), 0.994 (external test 1), 0.984 (external test 2), and 0.979 (external test 3), respectively. Meanwhile, the detection results of AI system showed a strong correlation with the diagnosis of cardiologists (cardiologist 1 (R=0.794, p<1e-3), cardiologist 2 (R=0.812, p<1e-3)). On the screening task, the AI system achieved AUCs of 0.894 (hold-out test) and 0.850 (external test). The screening performance of the AI system was better than that of the cardiologists (AI system (0.846) vs. cardiologist 1 (0.520) vs. cardiologist 2 (0.480)). Our study demonstrates the feasibility of an accurate, objective, easy-to-use, fast, and low-cost AI system for CA detection and screening. The system has the potential to be used by healthcare professionals, caregivers, and general users to assess CAs based on real-world ECG images.

Monte Carlo Neural Operator for Learning PDEs via Probabilistic Representation

Neural operators, which use deep neural networks to approximate the solution mappings of partial differential equation (PDE) systems, are emerging as a new paradigm for PDE simulation. The neural operators could be trained in supervised or unsupervised ways, i.e., by using the generated data or the PDE information. The unsupervised training approach is essential when data generation is costly or the data is less qualified (e.g., insufficient and noisy). However, its performance and efficiency have plenty of room for improvement. To this end, we design a new loss function based on the Feynman-Kac formula and call the developed neural operator Monte-Carlo Neural Operator (MCNO), which can allow larger temporal steps and efficiently handle fractional diffusion operators. Our analyses show that MCNO has advantages in handling complex spatial conditions and larger temporal steps compared with other unsupervised methods. Furthermore, MCNO is more robust with the perturbation raised by the numerical scheme and operator approximation. Numerical experiments on the diffusion equation and Navier-Stokes equation show significant accuracy improvement compared with other unsupervised baselines, especially for the vibrated initial condition and long-time simulation settings.

Target-Mounted Intelligent Reflecting Surface for Joint Location and Orientation Estimation

Intelligent reflecting surface (IRS) has been widely recognized as an efficient technique to reconfigure the electromagnetic environment in favor of wireless communication performance. In this paper, we propose a new application of IRS for device-free target sensing via joint location and orientation estimation. In particular, different from the existing works that use IRS as an additional anchor node for localization/sensing, we consider mounting IRS on the sensing target, whereby estimating the IRS's location and orientation as that of the target by leveraging IRS's controllable signal reflection. To this end, we first propose a tensor-based method to acquire essential angle information between the IRS and the sensing transmitter as well as a set of distributed sensing receivers. Next, based on the estimated angle information, we formulate two optimization problems to estimate the location and orientation of the IRS/target, respectively, and obtain the locally optimal solutions to them by invoking two iterative algorithms, namely, gradient descent method and manifold optimization. In particular, we show that the orientation estimation problem admits a closed-form solution in a special case that usually holds in practice. Furthermore, theoretical analysis is conducted to draw essential insights into the proposed sensing system design and performance. Simulation results verify our theoretical analysis and demonstrate that the proposed methods can achieve high estimation accuracy which is close to the theoretical bound.