Task-Balanced Distillation for Object Detection

Mainstream object detectors are commonly constituted of two sub-tasks, including classification and regression tasks, implemented by two parallel heads. This classic design paradigm inevitably leads to inconsistent spatial distributions between classification score and localization quality (IOU). Therefore, this paper alleviates this misalignment in the view of knowledge distillation. First, we observe that the massive teacher achieves a higher proportion of harmonious predictions than the lightweight student. Based on this intriguing observation, a novel Harmony Score (HS) is devised to estimate the alignment of classification and regression qualities. HS models the relationship between two sub-tasks and is seen as prior knowledge to promote harmonious predictions for the student. Second, this spatial misalignment will result in inharmonious region selection when distilling features. To alleviate this problem, a novel Task-decoupled Feature Distillation (TFD) is proposed by flexibly balancing the contributions of classification and regression tasks. Eventually, HD and TFD constitute the proposed method, named Task-Balanced Distillation (TBD). Extensive experiments demonstrate the considerable potential and generalization of the proposed method. Specifically, when equipped with TBD, RetinaNet with ResNet-50 achieves 41.0 mAP under the COCO benchmark, outperforming the recent FGD and FRS.

Training Enhancement of Deep Learning Models for Massive MIMO CSI Feedback with Small Datasets

Accurate downlink channel state information (CSI) is vital to achieving high spectrum efficiency in massive MIMO systems. Existing works on the deep learning (DL) model for CSI feedback have shown efficient compression and recovery in frequency division duplex (FDD) systems. However, practical DL networks require sizeable wireless CSI datasets during training to achieve high model accuracy. To address this labor-intensive problem, this work develops an efficient training enhancement solution of DL-based feedback architecture based on a modest dataset by exploiting the complex CSI features, and augmenting CSI dataset based on domain knowledge. We first propose a spherical CSI feedback network, SPTM2-ISTANet+, which employs the spherical normalization framework to mitigate the effect of path loss variation. We exploit the trainable measurement matrix and residual recovery structure to improve the encoding efficiency and recovery accuracy. For limited CSI measurements, we propose a model-driven lightweight and universal augmentation strategy based on decoupling CSI magnitude and phase information, applying the circular shift in angular-delay domain, and randomizing the CSI phase to approximate phase distribution. Test results demonstrate the efficacy and efficiency of the proposed training strategy and feedback architecture for accurate CSI feedback under limited measurements.

Look Before You Leap: Safe Model-Based Reinforcement Learning with Human Intervention

Safety has become one of the main challenges of applying deep reinforcement learning to real world systems. Currently, the incorporation of external knowledge such as human oversight is the only means to prevent the agent from visiting the catastrophic state. In this paper, we propose MBHI, a novel framework for safe model-based reinforcement learning, which ensures safety in the state-level and can effectively avoid both "local" and "non-local" catastrophes. An ensemble of supervised learners are trained in MBHI to imitate human blocking decisions. Similar to human decision-making process, MBHI will roll out an imagined trajectory in the dynamics model before executing actions to the environment, and estimate its safety. When the imagination encounters a catastrophe, MBHI will block the current action and use an efficient MPC method to output a safety policy. We evaluate our method on several safety tasks, and the results show that MBHI achieved better performance in terms of sample efficiency and number of catastrophes compared to the baselines.

Learning-Based Phase Compression and Quantization for Massive MIMO CSI Feedback with Magnitude-Aided Information

In frequency-division duplexing (FDD) massive multiple-input multiple-output (MIMO) wireless systems, deep learning techniques are regarded as one of the most efficient solutions for CSI recovery. In recent times, to achieve better CSI magnitude recovery at base stations, advanced learning-based CSI feedback solutions decouple magnitude and phase recovery to fully leverage the strong correlation between current CSI magnitudes and those of previous time slots, uplink band, and near locations. However, the CSI phase recovery is a major challenge to further enhance the CSI recovery owing to its complicated patterns. In this letter, we propose a learning-based CSI feedback framework based on limited feedback and magnitude-aided information. In contrast to previous works, our proposed framework with a proposed loss function enables end-to-end learning to jointly optimize the CSI magnitude and phase recovery performance. Numerical simulations show that, the proposed loss function outperform alternate approaches for phase recovery over the overall CSI recovery in both indoor and outdoor scenarios. The performance of the proposed framework was also examined using different core layer designs.

Multi-Faceted Representation Learning with Hybrid Architecture for Time Series Classification

Time series classification problems exist in many fields and have been explored for a couple of decades. However, they still remain challenging, and their solutions need to be further improved for real-world applications in terms of both accuracy and efficiency. In this paper, we propose a hybrid neural architecture, called Self-Attentive Recurrent Convolutional Networks (SARCoN), to learn multi-faceted representations for univariate time series. SARCoN is the synthesis of long short-term memory networks with self-attentive mechanisms and Fully Convolutional Networks, which work in parallel to learn the representations of univariate time series from different perspectives. The component modules of the proposed architecture are trained jointly in an end-to-end manner and they classify the input time series in a cooperative way. Due to its domain-agnostic nature, SARCoN is able to generalize a diversity of domain tasks. Our experimental results show that, compared to the state-of-the-art approaches for time series classification, the proposed architecture can achieve remarkable improvements for a set of univariate time series benchmarks from the UCR repository. Moreover, the self-attention and the global average pooling in the proposed architecture enable visible interpretability by facilitating the identification of the contribution regions of the original time series. An overall analysis confirms that multi-faceted representations of time series aid in capturing deep temporal corrections within complex time series, which is essential for the improvement of time series classification performance. Our work provides a novel angle that deepens the understanding of time series classification, qualifying our proposed model as an ideal choice for real-world applications.

A Markovian Model-Driven Deep Learning Framework for Massive MIMO CSI Feedback

Forward channel state information (CSI) often plays a vital role in scheduling and capacity-approaching transmission optimization for massive multiple-input multiple-output (MIMO) communication systems. In frequency division duplex (FDD) massive MIMO systems, forwardlink CSI reconstruction at the transmitter relies critically on CSI feedback from receiving nodes and must carefully weigh the tradeoff between reconstruction accuracy and feedback bandwidth. Recent studies on the use of recurrent neural networks (RNNs) have demonstrated strong promises, though the cost of computation and memory remains high, for massive MIMO deployment. In this work, we exploit channel coherence in time to substantially improve the feedback efficiency. Using a Markovian model, we develop a deep convolutional neural network (CNN)-based framework MarkovNet to differentially encode forward CSI in time to effectively improve reconstruction accuracy. Furthermore, we explore important physical insights, including spherical normalization of input data and convolutional layers for feedback compression. We demonstrate substantial performance improvement and complexity reduction over the RNN-based work by our proposed MarkovNet to recover forward CSI estimates accurately. We explore additional practical consideration in feedback quantization, and show that MarkovNet outperforms RNN-based CSI estimation networks at a fraction of the computational cost.

Text Classification based on Multi-granularity Attention Hybrid Neural Network

Neural network-based approaches have become the driven forces for Natural Language Processing (NLP) tasks. Conventionally, there are two mainstream neural architectures for NLP tasks: the recurrent neural network (RNN) and the convolution neural network (ConvNet). RNNs are good at modeling long-term dependencies over input texts, but preclude parallel computation. ConvNets do not have memory capability and it has to model sequential data as un-ordered features. Therefore, ConvNets fail to learn sequential dependencies over the input texts, but it is able to carry out high-efficient parallel computation. As each neural architecture, such as RNN and ConvNets, has its own pro and con, integration of different architectures is assumed to be able to enrich the semantic representation of texts, thus enhance the performance of NLP tasks. However, few investigation explores the reconciliation of these seemingly incompatible architectures. To address this issue, we propose a hybrid architecture based on a novel hierarchical multi-granularity attention mechanism, named Multi-granularity Attention-based Hybrid Neural Network (MahNN). The attention mechanism is to assign different weights to different parts of the input sequence to increase the computation efficiency and performance of neural models. In MahNN, two types of attentions are introduced: the syntactical attention and the semantical attention. The syntactical attention computes the importance of the syntactic elements (such as words or sentence) at the lower symbolic level and the semantical attention is used to compute the importance of the embedded space dimension corresponding to the upper latent semantics. We adopt the text classification as an exemplifying way to illustrate the ability of MahNN to understand texts.

Multichannel CNN with Attention for Text Classification

Recent years, the approaches based on neural networks have shown remarkable potential for sentence modeling. There are two main neural network structures: recurrent neural network (RNN) and convolution neural network (CNN). RNN can capture long term dependencies and store the semantics of the previous information in a fixed-sized vector. However, RNN is a biased model and its ability to extract global semantics is restricted by the fixed-sized vector. Alternatively, CNN is able to capture n-gram features of texts by utilizing convolutional filters. But the width of convolutional filters restricts its performance. In order to combine the strengths of the two kinds of networks and alleviate their shortcomings, this paper proposes Attention-based Multichannel Convolutional Neural Network (AMCNN) for text classification. AMCNN utilizes a bi-directional long short-term memory to encode the history and future information of words into high dimensional representations, so that the information of both the front and back of the sentence can be fully expressed. Then the scalar attention and vectorial attention are applied to obtain multichannel representations. The scalar attention can calculate the word-level importance and the vectorial attention can calculate the feature-level importance. In the classification task, AMCNN uses a CNN structure to cpture word relations on the representations generated by the scalar and vectorial attention mechanism instead of calculating the weighted sums. It can effectively extract the n-gram features of the text. The experimental results on the benchmark datasets demonstrate that AMCNN achieves better performance than state-of-the-art methods. In addition, the visualization results verify the semantic richness of multichannel representations.

Human Activity Recognition based on Dynamic Spatio-Temporal Relations

Human activity, which usually consists of several actions, generally covers interactions among persons and or objects. In particular, human actions involve certain spatial and temporal relationships, are the components of more complicated activity, and evolve dynamically over time. Therefore, the description of a single human action and the modeling of the evolution of successive human actions are two major issues in human activity recognition. In this paper, we develop a method for human activity recognition that tackles these two issues. In the proposed method, an activity is divided into several successive actions represented by spatio temporal patterns, and the evolution of these actions are captured by a sequential model. A refined comprehensive spatio temporal graph is utilized to represent a single action, which is a qualitative representation of a human action incorporating both the spatial and temporal relations of the participant objects. Next, a discrete hidden Markov model is applied to model the evolution of action sequences. Moreover, a fully automatic partition method is proposed to divide a long-term human activity video into several human actions based on variational objects and qualitative spatial relations. Finally, a hierarchical decomposition of the human body is introduced to obtain a discriminative representation for a single action. Experimental results on the Cornell Activity Dataset demonstrate the efficiency and effectiveness of the proposed approach, which will enable long videos of human activity to be better recognized.