Algorithm-driven Advances for Scientific CT Instruments: From Model-based to Deep Learning-based Approaches

Multi-scale 3D characterization is widely used by materials scientists to further their understanding of the relationships between microscopic structure and macroscopic function. Scientific computed tomography (CT) instruments are one of the most popular choices for 3D non-destructive characterization of materials at length scales ranging from the angstrom-scale to the micron-scale. These instruments typically have a source of radiation that interacts with the sample to be studied and a detector assembly to capture the result of this interaction. A collection of such high-resolution measurements are made by re-orienting the sample which is mounted on a specially designed stage/holder after which reconstruction algorithms are used to produce the final 3D volume of interest. The end goal of scientific CT scans include determining the morphology,chemical composition or dynamic behavior of materials when subjected to external stimuli. In this article, we will present an overview of recent advances in reconstruction algorithms that have enabled significant improvements in the performance of scientific CT instruments - enabling faster, more accurate and novel imaging capabilities. In the first part, we will focus on model-based image reconstruction algorithms that formulate the inversion as solving a high-dimensional optimization problem involving a data-fidelity term and a regularization term. In the last part of the article, we will present an overview of recent approaches using deep-learning based algorithms for improving scientific CT instruments.

Model-based Reconstruction for Single Particle Cryo-Electron Microscopy

Single particle cryo-electron microscopy is a vital tool for 3D characterization of protein structures. A typical workflow involves acquiring projection images of a collection of randomly oriented particles, picking and classifying individual particle projections by orientation, and finally using the individual particle projections to reconstruct a 3D map of the electron density profile. The reconstruction is challenging because of the low signal-to-noise ratio of the data, the unknown orientation of the particles, and the sparsity of data especially when dealing with flexible proteins where there may not be sufficient data corresponding to each class to obtain an accurate reconstruction using standard algorithms. In this paper we present a model-based image reconstruction technique that uses a regularized cost function to reconstruct the 3D density map by assuming known orientations for the particles. Our method casts the reconstruction as minimizing a cost function involving a novel forward model term that accounts for the contrast transfer function of the microscope, the orientation of the particles and the center of rotation offsets. We combine the forward model term with a regularizer that enforces desirable properties in the volume to be reconstructed. Using simulated data, we demonstrate how our method can significantly improve upon the typically used approach.

Multi-resolution Data Fusion for Super-Resolution Electron Microscopy

Perhaps surprisingly, the total electron microscopy (EM) data collected to date is less than a cubic millimeter. Consequently, there is an enormous demand in the materials and biological sciences to image at greater speed and lower dosage, while maintaining resolution. Traditional EM imaging based on homogeneous raster-order scanning severely limits the volume of high-resolution data that can be collected, and presents a fundamental limitation to understanding physical processes such as material deformation, crack propagation, and pyrolysis. We introduce a novel multi-resolution data fusion (MDF) method for super-resolution computational EM. Our method combines innovative data acquisition with novel algorithmic techniques to dramatically improve the resolution/volume/speed trade-off. The key to our approach is to collect the entire sample at low resolution, while simultaneously collecting a small fraction of data at high resolution. The high-resolution measurements are then used to create a material-specific patch-library that is used within the "plug-and-play" framework to dramatically improve super-resolution of the low-resolution data. We present results using FEI electron microscope data that demonstrate super-resolution factors of 4x, 8x, and 16x, while substantially maintaining high image quality and reducing dosage.

Plug-and-Play Priors for Bright Field Electron Tomography and Sparse Interpolation

Many material and biological samples in scientific imaging are characterized by non-local repeating structures. These are studied using scanning electron microscopy and electron tomography. Sparse sampling of individual pixels in a 2D image acquisition geometry, or sparse sampling of projection images with large tilt increments in a tomography experiment, can enable high speed data acquisition and minimize sample damage caused by the electron beam. In this paper, we present an algorithm for electron tomographic reconstruction and sparse image interpolation that exploits the non-local redundancy in images. We adapt a framework, termed plug-and-play (P&P) priors, to solve these imaging problems in a regularized inversion setting. The power of the P&P approach is that it allows a wide array of modern denoising algorithms to be used as a "prior model" for tomography and image interpolation. We also present sufficient mathematical conditions that ensure convergence of the P&P approach, and we use these insights to design a new non-local means denoising algorithm. Finally, we demonstrate that the algorithm produces higher quality reconstructions on both simulated and real electron microscope data, along with improved convergence properties compared to other methods.