Alert button
Picture for Mehmet Akçakaya

Mehmet Akçakaya

Alert button

On the Robustness of deep learning-based MRI Reconstruction to image transformations

Nov 21, 2022
Jinghan Jia, Mingyi Hong, Yimeng Zhang, Mehmet Akçakaya, Sijia Liu

Figure 1 for On the Robustness of deep learning-based MRI Reconstruction to image transformations
Figure 2 for On the Robustness of deep learning-based MRI Reconstruction to image transformations
Figure 3 for On the Robustness of deep learning-based MRI Reconstruction to image transformations
Figure 4 for On the Robustness of deep learning-based MRI Reconstruction to image transformations

Although deep learning (DL) has received much attention in accelerated magnetic resonance imaging (MRI), recent studies show that tiny input perturbations may lead to instabilities of DL-based MRI reconstruction models. However, the approaches of robustifying these models are underdeveloped. Compared to image classification, it could be much more challenging to achieve a robust MRI image reconstruction network considering its regression-based learning objective, limited amount of training data, and lack of efficient robustness metrics. To circumvent the above limitations, our work revisits the problem of DL-based image reconstruction through the lens of robust machine learning. We find a new instability source of MRI image reconstruction, i.e., the lack of reconstruction robustness against spatial transformations of an input, e.g., rotation and cutout. Inspired by this new robustness metric, we develop a robustness-aware image reconstruction method that can defend against both pixel-wise adversarial perturbations as well as spatial transformations. Extensive experiments are also conducted to demonstrate the effectiveness of our proposed approaches.

* Accepted as TSRML'22 Paper 
Viaarxiv icon

Accelerated MRI With Deep Linear Convolutional Transform Learning

Apr 17, 2022
Hongyi Gu, Burhaneddin Yaman, Steen Moeller, Il Yong Chun, Mehmet Akçakaya

Figure 1 for Accelerated MRI With Deep Linear Convolutional Transform Learning
Figure 2 for Accelerated MRI With Deep Linear Convolutional Transform Learning
Figure 3 for Accelerated MRI With Deep Linear Convolutional Transform Learning
Figure 4 for Accelerated MRI With Deep Linear Convolutional Transform Learning

Recent studies show that deep learning (DL) based MRI reconstruction outperforms conventional methods, such as parallel imaging and compressed sensing (CS), in multiple applications. Unlike CS that is typically implemented with pre-determined linear representations for regularization, DL inherently uses a non-linear representation learned from a large database. Another line of work uses transform learning (TL) to bridge the gap between these two approaches by learning linear representations from data. In this work, we combine ideas from CS, TL and DL reconstructions to learn deep linear convolutional transforms as part of an algorithm unrolling approach. Using end-to-end training, our results show that the proposed technique can reconstruct MR images to a level comparable to DL methods, while supporting uniform undersampling patterns unlike conventional CS methods. Our proposed method relies on convex sparse image reconstruction with linear representation at inference time, which may be beneficial for characterizing robustness, stability and generalizability.

* To be published in 2022 44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) 
Viaarxiv icon

Physics-Driven Deep Learning for Computational Magnetic Resonance Imaging

Mar 23, 2022
Kerstin Hammernik, Thomas Küstner, Burhaneddin Yaman, Zhengnan Huang, Daniel Rueckert, Florian Knoll, Mehmet Akçakaya

Figure 1 for Physics-Driven Deep Learning for Computational Magnetic Resonance Imaging
Figure 2 for Physics-Driven Deep Learning for Computational Magnetic Resonance Imaging
Figure 3 for Physics-Driven Deep Learning for Computational Magnetic Resonance Imaging
Figure 4 for Physics-Driven Deep Learning for Computational Magnetic Resonance Imaging

Physics-driven deep learning methods have emerged as a powerful tool for computational magnetic resonance imaging (MRI) problems, pushing reconstruction performance to new limits. This article provides an overview of the recent developments in incorporating physics information into learning-based MRI reconstruction. We consider inverse problems with both linear and non-linear forward models for computational MRI, and review the classical approaches for solving these. We then focus on physics-driven deep learning approaches, covering physics-driven loss functions, plug-and-play methods, generative models, and unrolled networks. We highlight domain-specific challenges such as real- and complex-valued building blocks of neural networks, and translational applications in MRI with linear and non-linear forward models. Finally, we discuss common issues and open challenges, and draw connections to the importance of physics-driven learning when combined with other downstream tasks in the medical imaging pipeline.

Viaarxiv icon

Unsupervised Deep Learning Methods for Biological Image Reconstruction

May 17, 2021
Mehmet Akçakaya, Burhaneddin Yaman, Hyungjin Chung, Jong Chul Ye

Figure 1 for Unsupervised Deep Learning Methods for Biological Image Reconstruction
Figure 2 for Unsupervised Deep Learning Methods for Biological Image Reconstruction
Figure 3 for Unsupervised Deep Learning Methods for Biological Image Reconstruction
Figure 4 for Unsupervised Deep Learning Methods for Biological Image Reconstruction

Recently, deep learning approaches have become the main research frontier for biological image reconstruction problems thanks to their high performance, along with their ultra-fast reconstruction times. However, due to the difficulty of obtaining matched reference data for supervised learning, there has been increasing interest in unsupervised learning approaches that do not need paired reference data. In particular, self-supervised learning and generative models have been successfully used for various biological imaging applications. In this paper, we overview these approaches from a coherent perspective in the context of classical inverse problems, and discuss their applications to biological imaging.

Viaarxiv icon

20-fold Accelerated 7T fMRI Using Referenceless Self-Supervised Deep Learning Reconstruction

May 12, 2021
Omer Burak Demirel, Burhaneddin Yaman, Logan Dowdle, Steen Moeller, Luca Vizioli, Essa Yacoub, John Strupp, Cheryl A. Olman, Kâmil Uğurbil, Mehmet Akçakaya

Figure 1 for 20-fold Accelerated 7T fMRI Using Referenceless Self-Supervised Deep Learning Reconstruction
Figure 2 for 20-fold Accelerated 7T fMRI Using Referenceless Self-Supervised Deep Learning Reconstruction
Figure 3 for 20-fold Accelerated 7T fMRI Using Referenceless Self-Supervised Deep Learning Reconstruction
Figure 4 for 20-fold Accelerated 7T fMRI Using Referenceless Self-Supervised Deep Learning Reconstruction

High spatial and temporal resolution across the whole brain is essential to accurately resolve neural activities in fMRI. Therefore, accelerated imaging techniques target improved coverage with high spatio-temporal resolution. Simultaneous multi-slice (SMS) imaging combined with in-plane acceleration are used in large studies that involve ultrahigh field fMRI, such as the Human Connectome Project. However, for even higher acceleration rates, these methods cannot be reliably utilized due to aliasing and noise artifacts. Deep learning (DL) reconstruction techniques have recently gained substantial interest for improving highly-accelerated MRI. Supervised learning of DL reconstructions generally requires fully-sampled training datasets, which is not available for high-resolution fMRI studies. To tackle this challenge, self-supervised learning has been proposed for training of DL reconstruction with only undersampled datasets, showing similar performance to supervised learning. In this study, we utilize a self-supervised physics-guided DL reconstruction on a 5-fold SMS and 4-fold in-plane accelerated 7T fMRI data. Our results show that our self-supervised DL reconstruction produce high-quality images at this 20-fold acceleration, substantially improving on existing methods, while showing similar functional precision and temporal effects in the subsequent analysis compared to a standard 10-fold accelerated acquisition.

Viaarxiv icon

Improved Simultaneous Multi-Slice Functional MRI Using Self-supervised Deep Learning

May 10, 2021
Omer Burak Demirel, Burhaneddin Yaman, Logan Dowdle, Steen Moeller, Luca Vizioli, Essa Yacoub, John Strupp, Cheryl A. Olman, Kâmil Uğurbil, Mehmet Akçakaya

Figure 1 for Improved Simultaneous Multi-Slice Functional MRI Using Self-supervised Deep Learning
Figure 2 for Improved Simultaneous Multi-Slice Functional MRI Using Self-supervised Deep Learning
Figure 3 for Improved Simultaneous Multi-Slice Functional MRI Using Self-supervised Deep Learning
Figure 4 for Improved Simultaneous Multi-Slice Functional MRI Using Self-supervised Deep Learning

Functional MRI (fMRI) is commonly used for interpreting neural activities across the brain. Numerous accelerated fMRI techniques aim to provide improved spatiotemporal resolutions. Among these, simultaneous multi-slice (SMS) imaging has emerged as a powerful strategy, becoming a part of large-scale studies, such as the Human Connectome Project. However, when SMS imaging is combined with in-plane acceleration for higher acceleration rates, conventional SMS reconstruction methods may suffer from noise amplification and other artifacts. Recently, deep learning (DL) techniques have gained interest for improving MRI reconstruction. However, these methods are typically trained in a supervised manner that necessitates fully-sampled reference data, which is not feasible in highly-accelerated fMRI acquisitions. Self-supervised learning that does not require fully-sampled data has recently been proposed and has shown similar performance to supervised learning. However, it has only been applied for in-plane acceleration. Furthermore the effect of DL reconstruction on subsequent fMRI analysis remains unclear. In this work, we extend self-supervised DL reconstruction to SMS imaging. Our results on prospectively 10-fold accelerated 7T fMRI data show that self-supervised DL reduces reconstruction noise and suppresses residual artifacts. Subsequent fMRI analysis remains unaltered by DL processing, while the improved temporal signal-to-noise ratio produces higher coherence estimates between task runs.

Viaarxiv icon

On Instabilities of Conventional Multi-Coil MRI Reconstruction to Small Adverserial Perturbations

Feb 25, 2021
Chi Zhang, Jinghan Jia, Burhaneddin Yaman, Steen Moeller, Sijia Liu, Mingyi Hong, Mehmet Akçakaya

Figure 1 for On Instabilities of Conventional Multi-Coil MRI Reconstruction to Small Adverserial Perturbations
Figure 2 for On Instabilities of Conventional Multi-Coil MRI Reconstruction to Small Adverserial Perturbations
Figure 3 for On Instabilities of Conventional Multi-Coil MRI Reconstruction to Small Adverserial Perturbations
Figure 4 for On Instabilities of Conventional Multi-Coil MRI Reconstruction to Small Adverserial Perturbations

Although deep learning (DL) has received much attention in accelerated MRI, recent studies suggest small perturbations may lead to instabilities in DL-based reconstructions, leading to concern for their clinical application. However, these works focus on single-coil acquisitions, which is not practical. We investigate instabilities caused by small adversarial attacks for multi-coil acquisitions. Our results suggest that, parallel imaging and multi-coil CS exhibit considerable instabilities against small adversarial perturbations.

* To appear in Proceedings of the 29th Annual Meeting of ISMRM, 2021 
Viaarxiv icon

Scan-Specific MRI Reconstruction using Zero-Shot Physics-Guided Deep Learning

Feb 15, 2021
Burhaneddin Yaman, Seyed Amir Hossein Hosseini, Mehmet Akçakaya

Figure 1 for Scan-Specific MRI Reconstruction using Zero-Shot Physics-Guided Deep Learning
Figure 2 for Scan-Specific MRI Reconstruction using Zero-Shot Physics-Guided Deep Learning
Figure 3 for Scan-Specific MRI Reconstruction using Zero-Shot Physics-Guided Deep Learning
Figure 4 for Scan-Specific MRI Reconstruction using Zero-Shot Physics-Guided Deep Learning

Physics-guided deep learning (PG-DL) has emerged as a powerful tool for accelerated MRI reconstruction, while often necessitating a database of fully-sampled measurements for training. Recent self-supervised and unsupervised learning approaches enable training without fully-sampled data. However, a database of undersampled measurements may not be available in many scenarios, especially for scans involving contrast or recently developed sequences, necessitating new methodology for scan-specific PG-DL reconstructions. A main challenge for developing scan-specific PG-DL methods is the large number of parameters, making it prone to over-fitting. Moreover, database-trained models may not generalize to unseen measurements that differ in terms of SNR, image contrast or sampling pattern. In this work, we propose a zero-shot self-supervised learning approach to perform scan-specific PG-DL reconstruction to tackle these issues. The proposed approach splits available measurements for each scan into three disjoint sets. Two of these sets are used to enforce data consistency and define loss during training, while the last set is used to establish an early stopping criterion. In the presence of models pre-trained on a database, we show that the proposed approach can be adapted as scan-specific fine-tuning via transfer learning to further improve reconstruction quality.

Viaarxiv icon

Self-Supervised Physics-Guided Deep Learning Reconstruction For High-Resolution 3D LGE CMR

Nov 18, 2020
Burhaneddin Yaman, Chetan Shenoy, Zilin Deng, Steen Moeller, Hossam El-Rewaidy, Reza Nezafat, Mehmet Akçakaya

Figure 1 for Self-Supervised Physics-Guided Deep Learning Reconstruction For High-Resolution 3D LGE CMR
Figure 2 for Self-Supervised Physics-Guided Deep Learning Reconstruction For High-Resolution 3D LGE CMR
Figure 3 for Self-Supervised Physics-Guided Deep Learning Reconstruction For High-Resolution 3D LGE CMR
Figure 4 for Self-Supervised Physics-Guided Deep Learning Reconstruction For High-Resolution 3D LGE CMR

Late gadolinium enhancement (LGE) cardiac MRI (CMR) is the clinical standard for diagnosis of myocardial scar. 3D isotropic LGE CMR provides improved coverage and resolution compared to 2D imaging. However, image acceleration is required due to long scan times and contrast washout. Physics-guided deep learning (PG-DL) approaches have recently emerged as an improved accelerated MRI strategy. Training of PG-DL methods is typically performed in supervised manner requiring fully-sampled data as reference, which is challenging in 3D LGE CMR. Recently, a self-supervised learning approach was proposed to enable training PG-DL techniques without fully-sampled data. In this work, we extend this self-supervised learning approach to 3D imaging, while tackling challenges related to small training database sizes of 3D volumes. Results and a reader study on prospectively accelerated 3D LGE show that the proposed approach at 6-fold acceleration outperforms the clinically utilized compressed sensing approach at 3-fold acceleration.

Viaarxiv icon