Impacts of Color and Texture Distortions on Earth Observation Data in Deep Learning

Land cover classification and change detection are two important applications of remote sensing and Earth observation (EO) that have benefited greatly from the advances of deep learning. Convolutional and transformer-based U-net models are the state-of-the-art architectures for these tasks, and their performances have been boosted by an increased availability of large-scale annotated EO datasets. However, the influence of different visual characteristics of the input EO data on a model's predictions is not well understood. In this work we systematically examine model sensitivities with respect to several color- and texture-based distortions on the input EO data during inference, given models that have been trained without such distortions. We conduct experiments with multiple state-of-the-art segmentation networks for land cover classification and show that they are in general more sensitive to texture than to color distortions. Beyond revealing intriguing characteristics of widely used land cover classification models, our results can also be used to guide the development of more robust models within the EO domain.

Multi Target Tracking by Learning from Generalized Graph Differences

Formulating the multi object tracking problem as a network flow optimization problem is a popular choice. In this paper an efficient way of learning the weights of such a network is presented. It separates the problem into one embedding of feasible solutions into a one dimensional feature space and one optimization problem. The embedding can be learned using standard SGD type optimization without relying on an additional optimizations within each step. Training data is produced by performing small perturbations of ground truth tracks and representing them using generalized graph differences, which is an efficient way introduced to represent the difference between two graphs. The proposed method is evaluated on DukeMTMCT with competitive results.