SenseAI: Real-Time Inpainting for Electron Microscopy

Despite their proven success and broad applicability to Electron Microscopy (EM) data, joint dictionary-learning and sparse-coding based inpainting algorithms have so far remained impractical for real-time usage with an Electron Microscope. For many EM applications, the reconstruction time for a single frame is orders of magnitude longer than the data acquisition time, making it impossible to perform exclusively subsampled acquisition. This limitation has led to the development of SenseAI, a C++/CUDA library capable of extremely efficient dictionary-based inpainting. SenseAI provides N-dimensional dictionary learning, live reconstructions, dictionary transfer and visualization, as well as real-time plotting of statistics, parameters, and image quality metrics.

Subsampling Methods for Fast Electron Backscattered Diffraction Analysis

Despite advancements in electron backscatter diffraction (EBSD) detector speeds, the acquisition rates of 4-Dimensional (4D) EBSD data, i.e., a collection of 2-dimensional (2D) diffraction maps for every position of a convergent electron probe on the sample, is limited by the capacity of the detector. Such 4D data enables computation of, e.g., band contrast and Inverse Pole Figure (IPF) maps, used for material characterisation. In this work we propose a fast acquisition method of EBSD data through subsampling 2-D probe positions and inpainting. We investigate reconstruction of both band contrast and IPF maps using an unsupervised Bayesian dictionary learning approach, i.e., Beta process factor analysis. Numerical simulations achieve high quality reconstructed images from 10% subsampled data.

In silico Ptychography of Lithium-ion Cathode Materials from Subsampled 4-D STEM Data

High quality scanning transmission electron microscopy (STEM) data acquisition and analysis has become increasingly important due to the commercial demand for investigating the properties of complex materials such as battery cathodes; however, multidimensional techniques (such as 4-D STEM) which can improve resolution and sample information are ultimately limited by the beam-damage properties of the materials or the signal-to-noise ratio of the result. subsampling offers a solution to this problem by retaining high signal, but distributing the dose across the sample such that the damage can be reduced. It is for these reasons that we propose a method of subsampling for 4-D STEM, which can take advantage of the redundancy within said data to recover functionally identical results to the ground truth. We apply these ideas to a simulated 4-D STEM data set of a LiMnO2 sample and we obtained high quality reconstruction of phase images using 12.5% subsampling.

A Targeted Sampling Strategy for Compressive Cryo Focused Ion Beam Scanning Electron Microscopy

Cryo Focused Ion-Beam Scanning Electron Microscopy (cryo FIB-SEM) enables three-dimensional and nanoscale imaging of biological specimens via a slice and view mechanism. The FIB-SEM experiments are, however, limited by a slow (typically, several hours) acquisition process and the high electron doses imposed on the beam sensitive specimen can cause damage. In this work, we present a compressive sensing variant of cryo FIB-SEM capable of reducing the operational electron dose and increasing speed. We propose two Targeted Sampling (TS) strategies that leverage the reconstructed image of the previous sample layer as a prior for designing the next subsampling mask. Our image recovery is based on a blind Bayesian dictionary learning approach, i.e., Beta Process Factor Analysis (BPFA). This method is experimentally viable due to our ultra-fast GPU-based implementation of BPFA. Simulations on artificial compressive FIB-SEM measurements validate the success of proposed methods: the operational electron dose can be reduced by up to 20 times. These methods have large implications for the cryo FIB-SEM community, in which the imaging of beam sensitive biological materials without beam damage is crucial.

SIM-STEM Lab: Incorporating Compressed Sensing Theory for Fast STEM Simulation

Recently it has been shown that precise dose control and an increase in the overall acquisition speed of atomic resolution scanning transmission electron microscope (STEM) images can be achieved by acquiring only a small fraction of the pixels in the image experimentally and then reconstructing the full image using an inpainting algorithm. In this paper, we apply the same inpainting approach (a form of compressed sensing) to simulated, sub-sampled atomic resolution STEM images. We find that it is possible to significantly sub-sample the area that is simulated, the number of g-vectors contributing the image, and the number of frozen phonon configurations contributing to the final image while still producing an acceptable fit to a fully sampled simulation. Here we discuss the parameters that we use and how the resulting simulations can be quantifiably compared to the full simulations. As with any Compressed Sensing methodology, care must be taken to ensure that isolated events are not excluded from the process, but the observed increase in simulation speed provides significant opportunities for real time simulations, image classification and analytics to be performed as a supplement to experiments on a microscope to be developed in the future.

Compressive Scanning Transmission Electron Microscopy

Scanning Transmission Electron Microscopy (STEM) offers high-resolution images that are used to quantify the nanoscale atomic structure and composition of materials and biological specimens. In many cases, however, the resolution is limited by the electron beam damage, since in traditional STEM, a focused electron beam scans every location of the sample in a raster fashion. In this paper, we propose a scanning method based on the theory of Compressive Sensing (CS) and subsampling the electron probe locations using a line hop sampling scheme that significantly reduces the electron beam damage. We experimentally validate the feasibility of the proposed method by acquiring real CS-STEM data, and recovering images using a Bayesian dictionary learning approach. We support the proposed method by applying a series of masks to fully-sampled STEM data to simulate the expectation of real CS-STEM. Finally, we perform the real data experimental series using a constrained-dose budget to limit the impact of electron dose upon the results, by ensuring that the total electron count remains constant for each image.