Data-induced multiscale losses and efficient multirate gradient descent schemes

This paper investigates the impact of multiscale data on machine learning algorithms, particularly in the context of deep learning. A dataset is multiscale if its distribution shows large variations in scale across different directions. This paper reveals multiscale structures in the loss landscape, including its gradients and Hessians inherited from the data. Correspondingly, it introduces a novel gradient descent approach, drawing inspiration from multiscale algorithms used in scientific computing. This approach seeks to transcend empirical learning rate selection, offering a more systematic, data-informed strategy to enhance training efficiency, especially in the later stages.

Fast Antenna and Beam Switching Method for mmWave Handsets with Hand Blockage

Many operators have been bullish on the role of millimeter-wave (mmWave) communications in fifth-generation (5G) mobile broadband because of its capability of delivering extreme data speeds and capacity. However, mmWave comes with challenges related to significantly high path loss and susceptibility to blockage. Particularly, when mmWave communication is applied to a mobile terminal device, communication can be frequently broken because of rampant hand blockage. Although a number of mobile phone companies have suggested configuring multiple sets of antenna modules at different locations on a mobile phone to circumvent this problem, identifying an optimal antenna module and a beam pair by simultaneously opening multiple sets of antenna modules causes the problem of excessive power consumption and device costs. In this study, a fast antenna and beam switching method termed Fast-ABS is proposed. In this method, only one antenna module is used for the reception to predict the best beam of other antenna modules. As such, unmasked antenna modules and their corresponding beam pairs can be rapidly selected for switching to avoid the problem of poor quality or disconnection of communications caused by hand blockage. Thorough analysis and extensive simulations, which include the derivation of relevant Cram\'{e}r-Rao lower bounds, show that the performance of Fast-ABS is close to that of an oracle solution that can instantaneously identify the best beam of other antenna modules even in complex multipath scenarios. Furthermore, Fast-ABS is implemented on a software defined radio and integrated into a 5G New Radio physical layer. Over-the-air experiments reveal that Fast-ABS can achieve efficient and seamless connectivity despite hand blockage.

Phase Retrieval using Expectation Consistent Signal Recovery Algorithm based on Hypernetwork

Phase retrieval (PR) is an important component in modern computational imaging systems. Many algorithms have been developed over the past half century. Recent advances in deep learning have opened up a new possibility for robust and fast PR. An emerging technique, called deep unfolding, provides a systematic connection between conventional model-based iterative algorithms and modern data-based deep learning. Unfolded algorithms, powered by data learning, have shown remarkable performance and convergence speed improvement over the original algorithms. Despite their potential, most existing unfolded algorithms are strictly confined to a fixed number of iterations when employing layer-dependent parameters. In this study, we develop a novel framework for deep unfolding to overcome the existing limitations. Even if our framework can be widely applied to general inverse problems, we take PR as an example in the paper. Our development is based on an unfolded generalized expectation consistent signal recovery (GEC-SR) algorithm, wherein damping factors are left for data-driven learning. In particular, we introduce a hypernetwork to generate the damping factors for GEC-SR. Instead of directly learning a set of optimal damping factors, the hypernetwork learns how to generate the optimal damping factors according to the clinical settings, thus ensuring its adaptivity to different scenarios. To make the hypernetwork work adapt to varying layer numbers, we use a recurrent architecture to develop a dynamic hypernetwork, which generates a damping factor that can vary online across layers. We also exploit a self-attention mechanism to enhance the robustness of the hypernetwork. Extensive experiments show that the proposed algorithm outperforms existing ones in convergence speed and accuracy, and still works well under very harsh settings, that many classical PR algorithms unstable or even fail.

RSAC: Regularized Subspace Approximation Classifier for Lightweight Continuous Learning

Continuous learning seeks to perform the learning on the data that arrives from time to time. While prior works have demonstrated several possible solutions, these approaches require excessive training time as well as memory usage. This is impractical for applications where time and storage are constrained, such as edge computing. In this work, a novel training algorithm, regularized subspace approximation classifier (RSAC), is proposed to achieve lightweight continuous learning. RSAC contains a feature reduction module and classifier module with regularization. Extensive experiments show that RSAC is more efficient than prior continuous learning works and outperforms these works on various experimental settings.

An Interpretable Compression and Classification System: Theory and Applications

This study proposes a low-complexity interpretable classification system. The proposed system contains three main modules including feature extraction, feature reduction, and classification. All of them are linear. Thanks to the linear property, the extracted and reduced features can be inversed to original data, like a linear transform such as Fourier transform, so that one can quantify and visualize the contribution of individual features towards the original data. Also, the reduced features and reversibility naturally endure the proposed system ability of data compression. This system can significantly compress data with a small percent deviation between the compressed and the original data. At the same time, when the compressed data is used for classification, it still achieves high testing accuracy. Furthermore, we observe that the extracted features of the proposed system can be approximated to uncorrelated Gaussian random variables. Hence, classical theory in estimation and detection can be applied for classification. This motivates us to propose using a MAP (maximum a posteriori) based classification method. As a result, the extracted features and the corresponding performance have statistical meaning and mathematically interpretable. Simulation results show that the proposed classification system not only enjoys significant reduced training and testing time but also high testing accuracy compared to the conventional schemes.