Extreme Low Resolution Activity Recognition with Spatial-Temporal Attention Transfer

Extreme low-resolution(LR) activity recognition plays a vital role in privacy protection. In the meantime, remote target recognition is also critical, especially in surveillance cameras. In this problem, the information capacity of LR data is relatively rare. How to exploit high-resolution(HR) data for improving the accuracy of LR action recognition is a notable issue. In this work, we make full use of the HR information of separate spatial and temporal features to promote LR recognition by acquiring better attention. Experiments show that our proposed method can improve LR recognition accuracy up to 4.4\%. Moreover, related experiments are implemented in the well-known datasets (e.g. UCF101 and HMDB51). The results achieve state-of-the-art performance on 12*16 HMDB51.

Towards Building the Semantic Map from a Monocular Camera with a Multi-task Network

In many robotic applications, especially for the autonomous driving, understanding the semantic information and the geometric structure of surroundings are both essential. Semantic 3D maps, as a carrier of the environmental knowledge, are then intensively studied for their abilities and applications. However, it is still challenging to produce a dense outdoor semantic map from a monocular image stream. Motivated by this target, in this paper, we propose a method for large-scale 3D reconstruction from consecutive monocular images. First, with the correlation of underlying information between depth and semantic prediction, a novel multi-task Convolutional Neural Network (CNN) is designed for joint prediction. Given a single image, the network learns low-level information with a shared encoder and separately predicts with decoders containing additional Atrous Spatial Pyramid Pooling (ASPP) layers and the residual connection which merits disparities and semantic mutually. To overcome the inconsistency of monocular depth prediction for reconstruction, post-processing steps with the superpixelization and the effective 3D representation approach are obtained to give the final semantic map. Experiments are compared with other methods on both semantic labeling and depth prediction. We also qualitatively demonstrate the map reconstructed from large-scale, difficult monocular image sequences to prove the effectiveness and superiority.