Coop: Memory is not a Commodity

Tensor rematerialization allows the training of deep neural networks (DNNs) under limited memory budgets by checkpointing the models and recomputing the evicted tensors as needed. However, the existing tensor rematerialization techniques overlook the memory system in deep learning frameworks and implicitly assume that free memory blocks at different addresses are identical. Under this flawed assumption, discontiguous tensors are evicted, among which some are not used to allocate the new tensor. This leads to severe memory fragmentation and increases the cost of potential rematerializations. To address this issue, we propose to evict tensors within a sliding window to ensure all evictions are contiguous and are immediately used. Furthermore, we proposed cheap tensor partitioning and recomputable in-place to further reduce the rematerialization cost by optimizing the tensor allocation. We named our method Coop as it is a co-optimization of tensor allocation and tensor rematerialization. We evaluated Coop on eight representative DNNs. The experimental results demonstrate that Coop achieves up to $2\times$ memory saving and hugely reduces compute overhead, search latency, and memory fragmentation compared to the state-of-the-art baselines.

Farthest Streamline Sampling for the Uniform Distribution of Forearm Muscle Fiber Tracts from Diffusion Tensor Imaging

Background: Diffusion tensor imaging (DTI) has been used to characterize forearm muscle architecture. Since only uniform sampling is performed for seed points rather than fiber tracts, the tracts may be unevenly distributed in the muscle volume. Purpose: To reconstruct uniformly distributed fiber tracts in human forearm by filtering the tracts from DTI. Assessment: Farthest streamline sampling (FSS) was proposed for filtering and compared with two conventional methods, i.e., two-dimensional sampling and three-dimensional sampling. The uniform coverage performance of the methods was evaluated by streamline coverage (SC) and the coefficient of variation of streamline density (SDCV). Architectural parameters were calculated for 17 forearm muscles. Anatomical correctness was verified by 1. visually assessing the fiber orientation, 2. checking whether the architectural parameters were within physiological ranges, and 3. classifying the architectural types. Results: FSS had the highest SC (0.93+0.04) and the lowest SDCV (0.34+0.06) among the three methods (P<0.05). FSS reduced the sampling of long tracts (10% reduction in fiber length, P<0.05), and the architectural parameters were within physiological ranges (two parameters with P<0.05). The fiber orientation of the tractography was visually consistent with that of the cadaveric specimen. The architectural types of 16 muscles were correctly classified, except for the palmaris longus, which had a linear arrangement of fiber endpoints (R2=0.95+0.02, P<0.001). Data Conclusion: FSS reconstructed more muscle regions and uniformly distributed fiber tracts. The tracts were anatomically correct, indicating the validity of fiber tracts. Key Words: diffusion tensor imaging; forearm muscles; architectural properties

Image-Based Vehicle Classification by Synergizing Features from Supervised and Self-Supervised Learning Paradigms

This paper introduces a novel approach to leverage features learned from both supervised and self-supervised paradigms, to improve image classification tasks, specifically for vehicle classification. Two state-of-the-art self-supervised learning methods, DINO and data2vec, were evaluated and compared for their representation learning of vehicle images. The former contrasts local and global views while the latter uses masked prediction on multi-layered representations. In the latter case, supervised learning is employed to finetune a pretrained YOLOR object detector for detecting vehicle wheels, from which definitive wheel positional features are retrieved. The representations learned from these self-supervised learning methods were combined with the wheel positional features for the vehicle classification task. Particularly, a random wheel masking strategy was utilized to finetune the previously learned representations in harmony with the wheel positional features during the training of the classifier. Our experiments show that the data2vec-distilled representations, which are consistent with our wheel masking strategy, outperformed the DINO counterpart, resulting in a celebrated Top-1 classification accuracy of 97.2% for classifying the 13 vehicle classes defined by the Federal Highway Administration.

Human Biophysics as Network Weights: Conditional Generative Models for Ultra-fast Simulation

Simulations of biophysical systems have provided a huge contribution to our fundamental understanding of human physiology and remain a central pillar for developments in medical devices and human machine interfaces. However, despite their successes, such simulations usually rely on highly computationally expensive numerical modelling, which is often inefficient to adapt to new simulation parameters. This limits their use in dynamic models of human behavior, for example in modelling the electric fields generated by muscles in a moving arm. We propose the alternative approach to use conditional generative models, which can learn complex relationships between the underlying generative conditions whilst remaining inexpensive to sample from. As a demonstration of this concept, we present BioMime, a hybrid architecture that combines elements of deep latent variable models and conditional adversarial training to construct a generative model that can both transform existing data samples to reflect new modelling assumptions and sample new data from a conditioned distribution. We demonstrate that BioMime can learn to accurately mimic a complex numerical model of human muscle biophysics and then use this knowledge to continuously sample from a dynamically changing system in real-time. We argue that transfer learning approaches with conditional generative models are a viable solution for dynamic simulation with any numerical model.