Generative deep learning-enabled ultra-large field-of-view lens-free imaging

Advancements in high-throughput biomedical applications necessitate real-time, large field-of-view (FOV) imaging capabilities. Conventional lens-free imaging (LFI) systems, while addressing the limitations of physical lenses, have been constrained by dynamic, hard-to-model optical fields, resulting in a limited one-shot FOV of approximately 20 $mm^2$. This restriction has been a major bottleneck in applications like live-cell imaging and automation of microfluidic systems for biomedical research. Here, we present a deep-learning(DL)-based imaging framework - GenLFI - leveraging generative artificial intelligence (AI) for holographic image reconstruction. We demonstrate that GenLFI can achieve a real-time FOV over 550 $mm^2$, surpassing the current LFI system by more than 20-fold, and even larger than the world's largest confocal microscope by 1.76 times. The resolution is at the sub-pixel level of 5.52 $\mu m$, without the need for a shifting light source. The unsupervised learning-based reconstruction does not require optical field modeling, making imaging dynamic 3D samples (e.g., droplet-based microfluidics and 3D cell models) in complex optical fields possible. This GenLFI framework unlocks the potential of LFI systems, offering a robust tool to tackle new frontiers in high-throughput biomedical applications such as drug discovery.

Digital twin-assisted three-dimensional electrical capacitance tomography for multiphase flow imaging

Three-dimensional electrical capacitance tomography (3D-ECT) has shown promise for visualizing industrial multiphase flows. However, existing 3D-ECT approaches suffer from limited imaging resolution and lack assessment metrics, hampering their effectiveness in quantitative multiphase flow imaging. This paper presents a digital twin (DT)-assisted 3D-ECT, aiming to overcome these limitations and enhance our understanding of multiphase flow dynamics. The DT framework incorporates a 3D fluid-electrostatic field coupling model (3D-FECM) that digitally represents the physical 3D-ECT system, which enables us to simulate real multiphase flows and generate a comprehensive virtual multiphase flow 3D imaging dataset. Additionally, the framework includes a deep neural network named 3D deep back projection (3D-DBP), which learns multiphase flow features from the virtual dataset and enables more accurate 3D flow imaging in the real world. The DT-assisted 3D-ECT was validated through virtual and physical experiments, demonstrating superior image quality, noise robustness and computational efficiency compared to conventional 3D-ECT approaches. This research contributes to developing accurate and reliable 3D-ECT techniques and their implementation in multiphase flow systems across various industries.

Touch and deformation perception of soft manipulators with capacitive e-skins and deep learning

Tactile sensing in soft robots remains particularly challenging because of the coupling between contact and deformation information which the sensor is subject to during actuation and interaction with the environment. This often results in severe interference and makes disentangling tactile sensing and geometric deformation difficult. To address this problem, this paper proposes a soft capacitive e-skin with a sparse electrode distribution and deep learning for information decoupling. Our approach successfully separates tactile sensing from geometric deformation, enabling touch recognition on a soft pneumatic actuator subject to both internal (actuation) and external (manual handling) forces. Using a multi-layer perceptron, the proposed e-skin achieves 99.88\% accuracy in touch recognition across a range of deformations. When complemented with prior knowledge, a transformer-based architecture effectively tracks the deformation of the soft actuator. The average distance error in positional reconstruction of the manipulator is as low as 2.905$\pm$2.207 mm, even under operative conditions with different inflation states and physical contacts which lead to additional signal variations and consequently interfere with deformation tracking. These findings represent a tangible way forward in the development of e-skins that can endow soft robots with proprioception and exteroception.

Regularized Shallow Image Prior for Electrical Impedance Tomography

Untrained Neural Network Prior (UNNP) based algorithms have gained increasing popularity in tomographic imaging, as they offer superior performance compared to hand-crafted priors and do not require training. UNNP-based methods usually rely on deep architectures which are known for their excellent feature extraction ability compared to shallow ones. Contrary to common UNNP-based approaches, we propose a regularized shallow image prior method that combines UNNP with hand-crafted prior for Electrical Impedance Tomography (EIT). Our approach employs a 3-layer Multi-Layer Perceptron (MLP) as the UNNP in regularizing 2D and 3D EIT inversion. We demonstrate the influence of two typical hand-crafted regularizations when representing the conductivity distribution with shallow MLPs. We show considerably improved EIT image quality compared to conventional regularization algorithms, especially in structure preservation. The results suggest that combining the shallow image prior and the hand-crafted regularization can achieve similar performance to the Deep Image Prior (DIP) but with less architectural dependency and complexity of the neural network.

A Learning-Based 3D EIT Image Reconstruction Method

Deep learning has been widely employed to solve the Electrical Impedance Tomography (EIT) image reconstruction problem. Most existing physical model-based and learning-based approaches focus on 2D EIT image reconstruction. However, when they are directly extended to the 3D domain, the reconstruction performance in terms of image quality and noise robustness is hardly guaranteed mainly due to the significant increase in dimensionality. This paper presents a learning-based approach for 3D EIT image reconstruction, which is named Transposed convolution with Neurons Network (TN-Net). Simulation and experimental results show the superior performance and generalization ability of TN-Net compared with prevailing 3D EIT image reconstruction algorithms.

Impedance-optical Dual-modal Cell Culture Imaging with Learning-based Information Fusion

While Electrical Impedance Tomography (EIT) has found many biomedicine applications, a better resolution is needed to provide quantitative analysis for tissue engineering and regenerative medicine. This paper proposes an impedance-optical dual-modal imaging framework, which is mainly aimed at high-quality 3D cell culture imaging and can be extended to other tissue engineering applications. The framework comprises three components, i.e., an impedance-optical dual-modal sensor, the guidance image processing algorithm, and a deep learning model named multi-scale feature cross fusion network (MSFCF-Net) for information fusion. The MSFCF-Net has two inputs, i.e., the EIT measurement and a binary mask image generated by the guidance image processing algorithm, whose input is an RGB microscopic image. The network then effectively fuses the information from the two different imaging modalities and generates the final conductivity image. We assess the performance of the proposed dual-modal framework by numerical simulation and MCF-7 cell imaging experiments. The results show that the proposed method could significantly improve image quality, indicating that impedance-optical joint imaging has the potential to reveal the structural and functional information of tissue-level targets simultaneously.

MMV-Net: A Multiple Measurement Vector Network for Multi-frequency Electrical Impedance Tomography

Multi-frequency Electrical Impedance Tomography (mfEIT) is an emerging biomedical imaging modality to reveal frequency-dependent conductivity distributions in biomedical applications. Conventional model-based image reconstruction methods suffer from low spatial resolution, unconstrained frequency correlation and high computational cost. Deep learning has been extensively applied in solving the EIT inverse problem in biomedical and industrial process imaging. However, most existing learning-based approaches deal with the single-frequency setup, which is inefficient and ineffective when extended to address the multi-frequency setup. In this paper, we present a Multiple Measurement Vector (MMV) model based learning algorithm named MMV-Net to solve the mfEIT image reconstruction problem. MMV-Net takes into account the correlations between mfEIT images and unfolds the update steps of the Alternating Direction Method of Multipliers (ADMM) for the MMV problem. The non-linear shrinkage operator associated with the weighted l2_1 regularization term is generalized with a cascade of a Spatial Self-Attention module and a Convolutional Long Short-Term Memory (ConvLSTM) module to capture intra- and inter-frequency dependencies. The proposed MMVNet was validated on our Edinburgh mfEIT Dataset and a series of comprehensive experiments. All reconstructed results show superior image quality, convergence performance and noise robustness against the state of the art.