MRSN: Multi-Relation Support Network for Video Action Detection

Action detection is a challenging video understanding task, requiring modeling spatio-temporal and interaction relations. Current methods usually model actor-actor and actor-context relations separately, ignoring their complementarity and mutual support. To solve this problem, we propose a novel network called Multi-Relation Support Network (MRSN). In MRSN, Actor-Context Relation Encoder (ACRE) and Actor-Actor Relation Encoder (AARE) model the actor-context and actor-actor relation separately. Then Relation Support Encoder (RSE) computes the supports between the two relations and performs relation-level interactions. Finally, Relation Consensus Module (RCM) enhances two relations with the long-term relations from the Long-term Relation Bank (LRB) and yields a consensus. Our experiments demonstrate that modeling relations separately and performing relation-level interactions can achieve and outperformer state-of-the-art results on two challenging video datasets: AVA and UCF101-24.

Uncertainty-based Network for Few-shot Image Classification

The transductive inference is an effective technique in the few-shot learning task, where query sets update prototypes to improve themselves. However, these methods optimize the model by considering only the classification scores of the query instances as confidence while ignoring the uncertainty of these classification scores. In this paper, we propose a novel method called Uncertainty-Based Network, which models the uncertainty of classification results with the help of mutual information. Specifically, we first data augment and classify the query instance and calculate the mutual information of these classification scores. Then, mutual information is used as uncertainty to assign weights to classification scores, and the iterative update strategy based on classification scores and uncertainties assigns the optimal weights to query instances in prototype optimization. Extensive results on four benchmarks show that Uncertainty-Based Network achieves comparable performance in classification accuracy compared to state-of-the-art method.

Learning Class-level Prototypes for Few-shot Learning

Few-shot learning aims to recognize new categories using very few labeled samples. Although few-shot learning has witnessed promising development in recent years, most existing methods adopt an average operation to calculate prototypes, thus limited by the outlier samples. In this work, we propose a simple yet effective framework for few-shot classification, which can learn to generate preferable prototypes from few support data, with the help of an episodic prototype generator module. The generated prototype is meant to be close to a certain \textit{\targetproto{}} and is less influenced by outlier samples. Extensive experiments demonstrate the effectiveness of this module, and our approach gets a significant raise over baseline models, and get a competitive result compared to previous methods on \textit{mini}ImageNet, \textit{tiered}ImageNet, and cross-domain (\textit{mini}ImageNet $\rightarrow$ CUB-200-2011) datasets.

Channel Relationship Prediction with Forget-Update Module for Few-shot Classification

In this paper, we proposed a pipeline for inferring the relationship of each class in support set and a query sample using forget-update module. We first propose a novel architectural module called "channel vector sequence construction module", which boosts the performance of sequence-prediction-model-based few-shot classification methods by collecting the overall information of all support samples and a query sample. The channel vector sequence generated by this module is organized in a way that each time step of the sequence contains the information from the corresponding channel of all support samples and the query sample to be inferred. Channel vector sequence is obtained by a convolutional neural network and a fully connected network, and the spliced channel vector sequence is spliced of the corresponding channel vectors of support samples and a query sample in the original channel order. Also, we propose a forget-update module consisting of stacked forget-update blocks. The forget block modify the original information with the learned weights and the update block establishes a dense connection for the model. The proposed pipeline, which consists of channel vector sequence construction module and forget-update module, can infer the relationship between the query sample and support samples in few-shot classification scenario. Experimental results show that the pipeline can achieve state-of-the-art results on miniImagenet, CUB dataset, and cross-domain scenario.