Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Exploiting Elasticity in Tensor Ranks for Compressing Neural Networks

May 10, 2021
Jie Ran, Rui Lin, Hayden K. H. So, Graziano Chesi, Ngai Wong

Share this with someone who'll enjoy it:

Elasticities in depth, width, kernel size and resolution have been explored in compressing deep neural networks (DNNs). Recognizing that the kernels in a convolutional neural network (CNN) are 4-way tensors, we further exploit a new elasticity dimension along the input-output channels. Specifically, a novel nuclear-norm rank minimization factorization (NRMF) approach is proposed to dynamically and globally search for the reduced tensor ranks during training. Correlation between tensor ranks across multiple layers is revealed, and a graceful tradeoff between model size and accuracy is obtained. Experiments then show the superiority of NRMF over the previous non-elastic variational Bayesian matrix factorization (VBMF) scheme.

* 8 pages, 5 figures 

   Access Paper Source

Share this with someone who'll enjoy it: