DepthSeg: Depth prompting in remote sensing semantic segmentation

Remote sensing semantic segmentation is crucial for extracting detailed land surface information, enabling applications such as environmental monitoring, land use planning, and resource assessment. In recent years, advancements in artificial intelligence have spurred the development of automatic remote sensing semantic segmentation methods. However, the existing semantic segmentation methods focus on distinguishing spectral characteristics of different objects while ignoring the differences in the elevation of the different targets. This results in land cover misclassification in complex scenarios involving shadow occlusion and spectral confusion. In this paper, we introduce a depth prompting two-dimensional (2D) remote sensing semantic segmentation framework (DepthSeg). It automatically models depth/height information from 2D remote sensing images and integrates it into the semantic segmentation framework to mitigate the effects of spectral confusion and shadow occlusion. During the feature extraction phase of DepthSeg, we introduce a lightweight adapter to enable cost-effective fine-tuning of the large-parameter vision transformer encoder pre-trained by natural images. In the depth prompting phase, we propose a depth prompter to model depth/height features explicitly. In the semantic prediction phase, we introduce a semantic classification decoder that couples the depth prompts with high-dimensional land-cover features, enabling accurate extraction of land-cover types. Experiments on the LiuZhou dataset validate the advantages of the DepthSeg framework in land cover mapping tasks. Detailed ablation studies further highlight the significance of the depth prompts in remote sensing semantic segmentation.

Deep Double Self-Expressive Subspace Clustering

Deep subspace clustering based on auto-encoder has received wide attention. However, most subspace clustering based on auto-encoder does not utilize the structural information in the self-expressive coefficient matrix, which limits the clustering performance. In this paper, we propose a double self-expressive subspace clustering algorithm. The key idea of our solution is to view the self-expressive coefficient as a feature representation of the example to get another coefficient matrix. Then, we use the two coefficient matrices to construct the affinity matrix for spectral clustering. We find that it can reduce the subspace-preserving representation error and improve connectivity. To further enhance the clustering performance, we proposed a self-supervised module based on contrastive learning, which can further improve the performance of the trained network. Experiments on several benchmark datasets demonstrate that the proposed algorithm can achieve better clustering than state-of-the-art methods.