ExplainS2A: Explainable Spectral-Spatial Duality Model for Fast Transforming Sentinel-2 Image to AVIRIS-Level Hyperspectral Image

Mainstream optical satellites often acquire multispectral multi-resolution images, which have limited material identifiability compared to the HSIs. Thus, spectrally super-resolving the MSI into their hyperspectral counterparts greatly facilitates remote material identification and the downstream tasks. However, spectrally super-resolving the MSI into an HSI is often constrained by the multi-resolution nature of the sensor. Specifically, due to the presence of some LR bands in the MSI, the initial spectral super-resolution results often appear to be spatially blurry, resulting in an LR HSI. To overcome this bottleneck, we then leverage some HR band inherent in the acquired MSI to spatially guide the reconstruction procedure, thereby yielding the desired HR HSI. This fusion procedure elegantly coincides with a widely known spatial super-resolution problem in satellite remote sensing. Hence, we have reformulated the tough spectral super-resolution problem into a more widely investigated spatial super-resolution problem, referred to as the spectral-spatial duality theory. Accordingly, we propose ExplainS2A, consisting of a deep unfolding network and an explainable fusion network, that unifies spectral recovery and spatial fusion into a single explainable framework. Unlike conventional black-box models, ExplainS2A offers interpretability and operates as a linear-time algorithm. Remarkably, it can process a million-scale Sentinel-2 image in less than one second, yielding high-fidelity HSI over the same scene, and upgrades the blind source separation results. Although demonstrated on the Sentinel-2 and AVIRIS sensors, ExplainS2A also serves as a general framework applicable to various sensor pairs with different resolution configurations, and has experimentally demonstrated cross-region and cross-season generalization ability. Source codes: https://github.com/IHCLab/ExplainS2A.

COS2A: Conversion from Sentinel-2 to AVIRIS Hyperspectral Data Using Interpretable Algorithm With Spectral-Spatial Duality

The Sentinel-2 satellite, launched by the European Space Agency (ESA), offers extensive spatial coverage and has become indispensable in a wide range of remote sensing applications. However, it just has 12 spectral bands, making substances/objects identification less effective, not mentioning the varying spatial resolutions (10/20/60 m) across the 12 bands. If such a multi-resolution 12-band image can be computationally converted into a hyperspectral image with uniformly high resolution (i.e., 10 m), it significantly facilitates remote identification tasks. Though there are some spectral super-resolution methods, they did not address the multi-resolution issue on one hand, and, more seriously, they mostly focused on the CAVE-level hyperspectral image reconstruction (involving only 31 visible bands) on the other hand, greatly limiting their applicability in real-world remote sensing scenarios. We ambitiously aim to convert Sentinel-2 data directly into NASA's AVIRIS-level hyperspectral image (encompassing up to 172 visible and near-infrared (NIR) bands, after ignoring those absorption/corruption ones). For the first time, this paper solves this specific super-resolution problem (highly ill-posed), allowing all historical Sentinel-2 data to have their corresponding high-standard AVIRIS counterparts. We achieve so by customizing a novel algorithm that introduces deep unfolding regularization and Q-quadratic-norm regularization into the so-called convex/deep (CODE) small-data learning criterion. Based on the derived spectral-spatial duality, the proposed interpretable COS2A algorithm demonstrates superior spectral super-resolution results across diverse land cover types, as validated through extensive experiments.

Hyper-Restormer: A General Hyperspectral Image Restoration Transformer for Remote Sensing Imaging

The deep learning model Transformer has achieved remarkable success in the hyperspectral image (HSI) restoration tasks by leveraging Spectral and Spatial Self-Attention (SA) mechanisms. However, applying these designs to remote sensing (RS) HSI restoration tasks, which involve far more spectrums than typical HSI (e.g., ICVL dataset with 31 bands), presents challenges due to the enormous computational complexity of using Spectral and Spatial SA mechanisms. To address this problem, we proposed Hyper-Restormer, a lightweight and effective Transformer-based architecture for RS HSI restoration. First, we introduce a novel Lightweight Spectral-Spatial (LSS) Transformer Block that utilizes both Spectral and Spatial SA to capture long-range dependencies of input features map. Additionally, we employ a novel Lightweight Locally-enhanced Feed-Forward Network (LLFF) to further enhance local context information. Then, LSS Transformer Blocks construct a Single-stage Lightweight Spectral-Spatial Transformer (SLSST) that cleverly utilizes the low-rank property of RS HSI to decompose the feature maps into basis and abundance components, enabling Spectral and Spatial SA with low computational cost. Finally, the proposed Hyper-Restormer cascades several SLSSTs in a stepwise manner to progressively enhance the quality of RS HSI restoration from coarse to fine. Extensive experiments were conducted on various RS HSI restoration tasks, including denoising, inpainting, and super-resolution, demonstrating that the proposed Hyper-Restormer outperforms other state-of-the-art methods.