Multi-Resolution Data Fusion for Super Resolution Imaging

Applications in materials and biological imaging are limited by the ability to collect high-resolution data over large areas in practical amounts of time. One possible solution to this problem is to collect low-resolution data and interpolate to produce a high-resolution image. However, state-of-the-art super-resolution algorithms are typically designed for natural images, require aligned pairing of high and low-resolution training data for optimal performance, and do not directly incorporate a model of the imaging sensor. In this paper, we present a Multi-Resolution Data Fusion (MDF) algorithm for accurate interpolation of low-resolution SEM and TEM data by factors of 4x and 8x. This MDF interpolation algorithm uses small quantities of unpaired high-resolution data to learn an accurate prior model denoiser and balances this with a forward model agent based on a mismatched back-projector that maintains fidelity to measured data. Our method is based on Multi-Agent Consensus Equilibrium, a generalization of the Plug-and-Play method, and allows for interpolation at arbitrary resolutions without retraining. We present electron microscopy results at 4x and 8x interpolation factors that exhibit reduced artifacts relative to existing methods while maintaining fidelity to acquired data and accurately resolving sub-pixel-scale features.