MTD: Multi-Timestep Detector for Delayed Streaming Perception

Autonomous driving systems require real-time environmental perception to ensure user safety and experience. Streaming perception is a task of reporting the current state of the world, which is used to evaluate the delay and accuracy of autonomous driving systems. In real-world applications, factors such as hardware limitations and high temperatures inevitably cause delays in autonomous driving systems, resulting in the offset between the model output and the world state. In order to solve this problem, this paper propose the Multi- Timestep Detector (MTD), an end-to-end detector which uses dynamic routing for multi-branch future prediction, giving model the ability to resist delay fluctuations. A Delay Analysis Module (DAM) is proposed to optimize the existing delay sensing method, continuously monitoring the model inference stack and calculating the delay trend. Moreover, a novel Timestep Branch Module (TBM) is constructed, which includes static flow and adaptive flow to adaptively predict specific timesteps according to the delay trend. The proposed method has been evaluated on the Argoverse-HD dataset, and the experimental results show that it has achieved state-of-the-art performance across various delay settings.

A Dynamics Theory of Implicit Regularization in Deep Low-Rank Matrix Factorization

Implicit regularization is an important way to interpret neural networks. Recent theory starts to explain implicit regularization with the model of deep matrix factorization (DMF) and analyze the trajectory of discrete gradient dynamics in the optimization process. These discrete gradient dynamics are relatively small but not infinitesimal, thus fitting well with the practical implementation of neural networks. Currently, discrete gradient dynamics analysis has been successfully applied to shallow networks but encounters the difficulty of complex computation for deep networks. In this work, we introduce another discrete gradient dynamics approach to explain implicit regularization, i.e. landscape analysis. It mainly focuses on gradient regions, such as saddle points and local minima. We theoretically establish the connection between saddle point escaping (SPE) stages and the matrix rank in DMF. We prove that, for a rank-R matrix reconstruction, DMF will converge to a second-order critical point after R stages of SPE. This conclusion is further experimentally verified on a low-rank matrix reconstruction problem. This work provides a new theory to analyze implicit regularization in deep learning.

Alternating Deep Low Rank Approach for Exponential Function Reconstruction and Its Biomedical Magnetic Resonance Applications

Exponential function is a fundamental signal form in general signal processing and biomedical applications, such as magnetic resonance spectroscopy and imaging. How to reduce the sampling time of these signals is an important problem. Sub-Nyquist sampling can accelerate signal acquisition but bring in artifacts. Recently, the low rankness of these exponentials has been applied to implicitly constrain the deep learning network through the unrolling of low rank Hankel factorization algorithm. However, only depending on the implicit low rank constraint cannot provide the robust reconstruction, such as sampling rate mismatches. In this work, by introducing the explicit low rank prior to constrain the deep learning, we propose an Alternating Deep Low Rank approach (ADLR) that utilizes deep learning and optimization solvers alternately. The former solver accelerates the reconstruction while the latter one corrects the reconstruction error from the mismatch. The experiments on both general exponential functions and realistic biomedical magnetic resonance data show that, compared with the state-of-the-art methods, ADLR can achieve much lower reconstruction error and effectively alleviates the decrease of reconstruction quality with sampling rate mismatches.

Accelerated NMR Spectroscopy: Merge Optimization with Deep Learning

Multi-dimensional NMR spectroscopy is an invaluable biophysical tool in studies of structure, interactions, and dynamics of large molecules like proteins and nuclear acids. Non-uniform sampling is a powerful approach for shortening measurement time and increasing spectra resolution. Several methods have been established for spectra reconstruction from the undersampled data, typical approaches include model-based optimization and data-driven deep learning. The former is well theoretically grounded and provides high-quality spectra, while the latter has a huge advantage in reconstruction time potential and push further limits of spectra quality and analysis. Combining the merits of the two, we propose a model-inspired deep learning, for reliable, high-quality, and ultra-fast spectra reconstruction, exemplified by multi-dimensional spectra of several representative proteins. We demonstrate that the proposed network needs very few parameters, and shows very high robustness in respect to dissimilarity of the training and target data in the spectra size, type, and sampling level. This work can be considered as a proof-of-concept of merging optimization with deep learning in NMR spectroscopy.

pISTA-SENSE-ResNet for Parallel MRI Reconstruction

Magnetic resonance imaging has been widely applied in clinical diagnosis, however, is limited by its long data acquisition time. Although imaging can be accelerated by sparse sampling and parallel imaging, achieving promising reconstruction images with a fast reconstruction speed remains a challenge. Recently, deep learning approaches have attracted a lot of attention for its encouraging reconstruction results but without a proper interpretability. In this letter, to enable high-quality image reconstruction for the parallel magnetic resonance imaging, we design the network structure from the perspective of sparse iterative reconstruction and enhance it with the residual structure. The experimental results of a public knee dataset show that compared with the optimization-based method and the latest deep learning parallel imaging methods, the proposed network has less error in reconstruction and is more stable under different acceleration factors.

Accelerated Nuclear Magnetic Resonance Spectroscopy with Deep Learning

Nuclear magnetic resonance (NMR) spectroscopy serves as an indispensable tool in chemistry and biology but often suffers from long experimental time. We present a proof-of-concept of application of deep learning and neural network for high-quality, reliable, and very fast NMR spectra reconstruction from limited experimental data. We show that the neural network training can be achieved using solely synthetic NMR signal, which lifts the prohibiting demand for a large volume of realistic training data usually required in the deep learning approach.